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ABSTRACT 


A  primal  feasible  (all-integer)  integer  linear  programming 
algorithm  has  been  developed  and  programmed,  together  with  a  related 
procedure  for  obtaining  a  first  feasible  solution.  Once  a  feasible 
solution  is  found,  the  algorithm  maintains  feasibility  at  each  stage,  in 
contrast  to  other  algorithms  that  have  been  programmed  and  are 
currently  available.  These  other  algorithms  do  not  achieve  feasibility 
until  the  optimal  solution  is  reached.  The  primal  feasible  algorithm  is 
based  on  a  particular  way  of  applying  the  cutting  planes  previously 
developed  by  R.  E.  GOMORY,  and  on  a  specific  interpretation  of  their 
role. 

The  finiteness  of  convergence  has  been  established  for  two- 
dimensional  problems  but  not  for  the  general  case;  however,  there 
appears  to  be  at  least  computational  convergence  in  a  considerable 
fraction  of  the  cases. 

In  addition,  a  Generalized  Euclidean  Algorithm  for  finding  the 
greatest  common  divisor  for  more  than  two  numbers  is  defined.  The 
solution  of  systems  of  linear  diophantine  equations  is  presented  in 
terms  of  integer  linear  programming. 

Some  geometric  considerations  that  help  to  illuminate  the  working 
of  the  algorithm,  are  examined. 

Thesis  Advisor:  Dr.  John  D.  C.  Little 

Title:  Associate  Professor  of  Industrial  Management 
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CHAPTER  1 


INTRODUCTION 

1.  1  The  Integer  and  the  Mixed  Linear  Programming  Problems 

An  Integer  LP  problem  is  a  linear  programming  problem  in  which 
all  the  variables  are  constrained  to  take  non-negative  integer  values.  In 
a  mixed  LP  problem  some  of  the  variables  are  constrained  to  be  integer, 
while  the  rest  are  continuous.  Both  problems  have  in  common  that  at 
least  one  of  the  variables  is  discrete.  We  can  cover  both  problems 
by  talking  about  LP  with  discrete  variables,  or  discrete  LP. 

Discrete  LP  is  important  because  there  are  many  practical  prob¬ 
lems  in  which  some  of  the  variables  are  actually  discrete;  for  instance, 
problems  dealing  with  the  number  of  flights,  machines,  men,  or  some 
other  indivisible  ur.lt.  If  a  problem  of  this  type  is  attacked  by  solving 
the  LP  problem  as  non-discrete  and  by  adjusting  the  solution  thus  ob¬ 
tained  (so  that  it  will  fulfill  the  discreteness  requirements),  it  might 
occur  that  the  adjusted  solution  is  far  from  optimal  for  the  discrete 
problem.  In  many  situations  it  will  be  profitable  to  use  more  refined 
methods  so  that  the  optimal  solution  is  found. 

Another  reason  why  discrete  LP  is  important  is  because  logical 
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relations  may  be  expressed  by  means  of  Boolean  variables  (variables 
that  can  only  take  one  of  the  values  0,  1),  and  thus  some  logical  con¬ 
straints  can  be  incorporated  in  the  definition  of  a  problem.  There  are 
many  combinatorial  problems  that  can  be  formulated  as  discrete  LP 
problems  by  using  integer  and  Boolean  Variables. 

1.  2  Available  Integer  LP  Algorithms 

In  this  section  we  give  a  brief  description  of  the  integer  LP  al¬ 
gorithms  that  are  currently  (May,  1965)  available  in  the  general  liter¬ 
ature  of  Operations  Research  and  Management  Science.  Most  of  these 
algorithms  use,  in  one  way  or  another,  the  "cuts'*  originally  discovered 
by  R.  E.  GOMORY  (refs.  [4],  [5]and  [6].  These  "Gomory-cuts" 

(or  "Gomory  inequalities")  are  secondary  conotraints  that  are  implied 
by  the  set  of  original  inequalities  and  by  the  discreteness  of  the  vari¬ 
ables.  Gomory  inequalities  will  be  analyzed  in  the  next  chapter. 

The  first  algorithm  by  Gomory  [4]  ,  [  5]  uses  the  simplex  tech¬ 
nique  to  obtain  the  solution  of  the  non -discrete  problem.  If  any  vari¬ 
able  turns  out  to  have  a  non-integer  value  a  secondary  constraint  is 
generated.  When  this  constraint  is  added  it  puts  the  problem  in  a 
"dual  feasible,  primal  infeasible"  form,  and  the  dual  simplex  method 
is  used  to  obtain  a  new  optimal  solution.  If  there  are  still  some  vari¬ 
ables  with  non-integer  variables,  the  procedure  of  generating  a  cut  and 
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re-solving  is  repeated  until  an  all  integer  solution  is  found.  The  algorithm 
is  shown  to  converge  to  the  optimum  in  a  finite  number  of  steps. 

The  second  of  Gomory's  algorithms  [6]  uses  only  integer  coeffi¬ 
cients  to  express  all  the  inequalities  and  produces  all-integer  solutions 
throughout  the  process;  but  these  solutions  are  not  primal  feasible  because, 
together  with  the  cuts,  a  dual  simplex  method  is  used.  Thus,  no  feasible 
solution  is  obtained  until  the  optimum  is  found.  Convergence  is  assured 
if  certain  rules  are  followed. 

The  algorithm  by  GLENN  T.  MARTIN  [8  ]  is  similar  to  the  first  of 
Gomory's  algorithms,  with  the  difference  that  the  cuts  that  Martin  gener¬ 
ates  are  better  (they  go  "deeper")  and  so  he  uses  less  iterations  to  get  to 
the  optimum.  But  as  in  the  first  case,  no  feasible  all -integer  solution  is 
obtained  until  the  end. 

RICKARD  VAN  SLYKE  AND  ROGER  WETS  [  1 1  ]  have  approached 
the  problem  of  improving  performance  of  integer  LP  algorithms  by  study¬ 
ing  the  way  of  easily  generating  efficient  cutting  planes.  They  handle 
this  by  using  a  triangular  canonical  form.  A  brief  outline  of  their  algo¬ 
rithm  is  given  in  their  paper  and  the  relationship  with  Gomory's  algorithm 
[6  ]  is  examined. 

FRED  GLOVER  has  studied  in  [2  ]  the  effect  of  different  heuristics 
upon  the  performance  of  Gomory's  All-Integer  Integer  LP  Algorithm  ff>]  . 

In  particular,  he  has  developed  the  NOT  (New  Origin  Technique)  heuristic. 
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which  by  means  of  the  simplex  algorithm  provides  a  better  starting 
point  for  Gomory's  algorithm.  In  his  second  paper  [3  ]  ,  Glover  develops 
another  approach  for  solving  integer  LP  problems.  An  integer  matrix 
is  transformed  into  another  integer  matrix  that  exhibits  certain  substruc¬ 
ture  called  bounding  form.  This  substructure  is  operated  upon  and  lower 
bounds  are  found  for  a  subset  of  the  variables.  This  information  and  the 
successively  derived  problem  matrices  are  used  in  conjunction  in  order 
to  guarantee  finite  convergence  for  the  solution  of  the  original  problem. 
Again,  no  feasible  solution  is  obtained  until  the  end. 

More  recently,  G.  L.  THOMPSON  [  10  ]  has  developed  a  solution 
enumeration  approach  that  is  characterized  by:  a)  use  of  the  number 
theoretic  properties  of  the  objective  function  to  obtain  a  convenient 
transformation  of  variables,  b)  application  of  a  multidimensional  search 
procedure  whose  memory  requirements  vary  only  linearly  with  the  size 
-of  the  problem.  Because  of  the  way  in  which  this  search  is  conducted, 
a  feasible  solution  is  found  only  when  an  optimal  solution  is  found.  His 
method  seems  to  be  more  efficient  than  Gomory's  [6  ]  ;  furthermore,  it 
can  also  be  applied  to  mixed  integer  LP  problems. 

Thus  no  algorithm,  of  the  ones  presented  so  far,  works  on  the 
principle  of  improving  upon  feasible  integer  solutions  until  the  optimal 
solution  is  found.  ADI  BEN-ISRAEL  and  A.  CHARNES  [  1  ]  propose  a 
Direct  Algorithm  that  has  this  motivation.  Their  algorithm  has  in  commoi 
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with  ours  the  fact  that  they  use  the  type  of  Gomory  cuts  which  we  call 
"non-zero  natural  inequalities".  However,  the  Direct  Algorithm  has  a 


very  important  shortcoming,  which  is  that  at  certain  stages  an  "Auxiliary 
Problem"  has  to  be  solved.  This  auxiliary  problem  is  an  integer  LP 
problem  that  might  be  as  large  as  the  original  problem;  but  they  do  not 
offer  any  systematic  method  for  solving  it,  other  than  obtaining  the  solu> 

I 

tion  of  its  continuous  counterpart  and  "interpolating"  from  itl  an  integer 


solution.  The  process  of  interpolating  is  not  explained.  Thp  Direct 
Algorithm  does  not  seem  to  be  practical  except  for  small  problems;  as 
a  matter  of  fact,  no  computational  results  are  reported. 


1.  3  Our  Results 

» 

A)  A  primal  feasible  (all-integer)  integer  LP  algorithm  has  been 

developed  and  programmed;  it  has  the  important  characteristic  that  at 

i 

any  stage  of  the  computation  one  always  has  a  feasible  integdr  solution. 
(Chapter  2) 


B)  A  procedure  for  obtaining  a  first  feasible  solution  has  also  been 
developed  and  programmed.  In  this  formulalation  we  do  not  need  addi¬ 
tional  variables  or  constraints  as  is  usually  necessary.  Of  special  interest 
is  a  technique  that  takes  advantage  of  any  original  equalities  in  order  to 
reduce  the  size  of  the  problem.  (Chapter  3) 


C)  A  proof  of  finite  convergence  of  the  primal  algorithm,  when 
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applied  to  two-dimensional  problems,  has  been  found.  (Chapter  4, 
section  4.  2) 

D)  We  have  not  established  the  finite  convergence  of  the  algorithm 
in  the  general  case.  Although  this  is  disappointing,  computational 
experience  with  other  algorithms  that  are  known  to  converge  is  not 
entirely  satisfactory.  A  certain  fraction  of  problems  tried  do  not  con¬ 
verge  in  any  tolerable  amount  of  time.  From  a  practical  point  of  view, 
then,  the  actual  performance  of  an  algorithm  on  the  computer  would 
appear  to  be  its  most  important  test.  In  this  respect,  the  experience 

so  far  is  that  our  algorithm  is  competitive  with  other  algorithms. 
Because  of  the  convergence  difficulty  that  exists  with  all  integer  LP 
algorithms,  the  primal  feasible  property  seems  particularly  desirable. 
There  is  no  other  programmed  primal  feasible  algorithm  that  we  know 
of.  (Chapter  4,  sections  4.  3  and  4.  4) 

E)  A  class  of  Generalized  Euclidean  Algorithms,  for  finding  the 
greatest  common  divisor  of  more  than  two  numbers,  is  defined.  Con¬ 
cepts  of  integer  LP  are  applied  to  the  problem  of  obtaining  the  general 
solution,  and  of  generating  non-negative  solutions,  of  systems  of  linear 
diophantine  equations.  (Chapter  5) 

F)  Geometric  considerations  that  throw  light  on  the  workings  of 
the  algorithm  are  examined.  Of  special  interest  is  a  technique  (inverse 
pivoting)  for  expressing  the  relevant  Gomory  cuts  explicitly  in  terms  of 
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the  original  variables  at  each  stage  of  the  algorithm.  (Chapter  6) 

G)  Finally,  some  specific  extensions  are  suggested.  (Chapter  7) 
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CHAPTER  2 


A  PRIMAL  FEASIBLE  ALGORITHM 


It  is  the  purpose  of  the  present  chapter  to  develop  a  primal  feasible 
algorithm  for  solving  the  integer  LP  problem.  Before  giving  the  outline 
of  the  basic  algorithm  we  will  introduce  the  necessary  notation,  present 
some  basic  properties  of  the  tableau  representation  and  discuss  the  spe¬ 
cial  constraints  called  "GOMORY  inequalities",  which  are  essential  for 
our  algorithm. 

2.  1  The  Problem.  Notation 

Our  notation  will  be  similar  to  the  one  used  by  Gomory  in 
references  [5  ]  and  [6]. 

We  will  consider  the  problem  of  finding  integers  x.  (j  =  1,  . . . ,  n) 

J 

that  maximize 


n 


j=l 


(2.  1.  1) 


subject  to 
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and  to 

x.  ^  0  j  *  1.....  n  (2.  1.3) 

J 

where  all  the  coefficients  and  constants  are  integers. 

By  introducing  the  non-negative  slack  variables  s.  and  by  using  the 

parametric  variables  t.  we  can  express  the  problem  given  by  (2,  1.  1), 

J 

(2.  1.  2)  and  (2.  1.  3)  as  follows: 

Maximize  z 


x  —  -  1  (-t.)  j  =  1, . . . ,  n 

J  J 

where  the  s^  and  the  x^  are  non-negative  integer  variables,  z  is  an 
unsigned  integer  variable,  and  all  the  coefficients  and  constants  are 
integers. 
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This  is  equivalent  to  solving  for  t.^  in  the  iO  equation  and 
replacing  this  result  for  t^  into  the  other  equations.  (The  marginal 
labels  "s.q1*  and  "t.^"  exchange  places  on  the  outside  of  the  tableau). 
In  vector  notation,  formulas  (2.  2.  1)  become 


jO  =  ~aj0  (1/aiO,jO) 


°j  =a  j  "aj0  (aiO,j/aiO,jO) 


j  =  0,  1,. . .  ,n;  j#  jO 


(2.  2.  2) 


Our  basic  algorithm  will  always  pivot  on  rows  that  will  be  added  |  , 

I'  |' 

to  the  bottom  of  the  tableau;  so  that  the  vertical  set  of  marginal  labels 

(i.  e.  z  to  x  )  will  always  remain  the  same, 
n 

The  current  solution  is  the  vector  of  values  that  are  obtained  if 
every  parametric  variable  is  set  equal  to  zero;  in  other  words,  it  is  the 
column  vector  a  =  (art  _,  a a  ,  ).  The  value  of  the  solution  . 
is  the  value  z  ,  that  is,  aQ  q  •  A  tableau  is  in  primal  feasible  form 
(and  the  sojiltion  is  said  to  be  primal  feasible,  or  simply,  feasible)  if  i 

V  '■ 

every  non-negative  variable  has  a  non -negative  value,  that  is,  if 
a,  5  0,  ip  1,.,,,  m+n.  A  tableau  is  said  to  be  in  dual  feasible 

lf  0 

form  if  a  5  0,  j  =  1, . . . ,  n  when  we  are  maximizing  z  ,  or  if 
0,  j 

a^  ^  <  0  ,  when  we  are  minimizing  z. 


A  column  a  .  is  lexicographically  positive  (a.>  0)  if  its  first  ele- 
J  J 

ment,  counting  from  the  top  down,  is  positive.  Column  ; .  is  (lexico-) 

larger  than  column  a.  if  a.  -a  .>  0  .  From  now,  when  we  refer  to  the 

1  J  1 
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sign  of  column,  unless  otherwise  stated,  it  is  to  be  understood  in  this 
lexicographical  sense. 

A  tableau  is  said  to  be  in  lexico-dual  feasible  form  if  every  column 
is  positive  (except,  perhaps,  column  zero)  when  we  are  maximizing,  or 
if  every  column  is  negative  when  we  are  minimizing. 

If  the  tableau  is  both  primal  and  dual  feasible  it  means  that  the 
objective  function  has  attained  its  optimal  value  and  the  current  solution 
is  one  of  the?  optimal  solutions  (see  Appendix  A).  If  the  tableau  is  both 
primal  feasible  and  lexico-dual  feasible  it  means  that  the  lexico -optimal 
solution  has  been  found.  That  is,  the  lexico-largest  feasible  solution 
has  been  found  if  we  are  maximizing  and  the  lexico -smallest  if  we  are 
minimizing  (see  Appendix  A).  In  either  case  we  say  that  optimality  has 
been  shown  or  proved;  in  the  latter  case  we  also  say  that  lexico  -optimality 
has  been  proved. 

The  purpose  of  pivoting  is  to  bring  the  tableau  into  an  optimal 
form.  We  may  start  with  a  primal  feasible  form  and  try  to  attain  dual 
feasibility  by  appropriate  pivoting  (while  maintaining  primal  feasibility), 
or  we  may  begin  with  a  dual  feasible  form  and  try  to  attain  primal 
feasibility  (while  maintaining  dual  feasibility).  The  first  technique  is 
usually  called  a  (primal)  simplex  method,  and  the  second  one  a  dual 
simplex  method.  If  instead  of  trying  to  attain,  or  to  maintain,  plain 
dual  feasibility,  we  try  to  attain,  or  to  maintain,  lexico-dual  feasibility. 
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we  will  have  a  lexicographical  simplex  method,  and  a  lexicographical 


dual  simplex  method  respectively. 

In  Appendix  A  we  present  an  example  that  shows  that  it  is  necessary 
to  use  a  lexicographical  simplex  method  in  our  basic  algorithm,  rather 
than  a  "plain"  simplex  method.  Whenever  we  mention  the  simplex 
method  in  the  rest  of  this  paper,  we  will  be  referring  to  the  lexicographical 
simplex  method,  unless  otherwise  stated.  Also,  we  will  usually  assume 
that  the  objective  is  to  maximize. 


If  the  pivoting  element  is  a^  ^  ,  row  iO  is  called  the  pivoting 
row  and  column  jO  is  the  pivoting  column  (column  zero  is  not  considered 
for  pivoting). 


When  using  the  simplex  method  we  will  pivot  on  negative  columns 
if  we  are  maximizing,  and  we  will  pivot  on  positive  columns  if  minimizing. 
The  reason  for  this  will  be  clear  below. 


Assume  we  have  chosen  a  column,  jO  ,  for  pivoting;  the  simplex 
method  criterion  for  choosing  a  pivoting  row  iO  ,  is  then  the  following: 


Let  I  be  the  set  of  indices  i  ,  i*  0 
iO  so  that: 


such  that  a.  .~~>0  .  Choose 

i»  jO 


(a...  „/a. „  )  =  min  {a.  „/a.  ..  } 

'  iO,  0  iO,  jO  .eI  i,  0'  i,  jO 


(2.  2.3) 


If  the  set  I  is  empty  and  we  are  minimizing,  it  means  that  the 
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objective  function  is  unbounded.  If  we  are  maximizing,  it  eithei  means 
that  the  objective  function  is  unbounded  (if  a  <  0  ),  and/or  that  some 

U#  jv 

of  the  problem  variables  are  unbounded  (for  further  discussion  see  section 
4.  1  and  Appendix  A). 

The  simplex  criterion  for  pivoting  guarantees  that  the  solution  will 
stay  primal  feasible  if  it  is  already  primal  feasible. 

If  we  are  maximizing,  the  column  vector  a  will  be  (lexico¬ 
graphically)  non-decreasing  because  a.^  q  ^0  ,  a.^  ^  >  0  ,  <  0 

and  from  (2.  2.  2) 

f 

Q  '  =  a  -  a.  (a.„  / a..  ..)  ^  a  „ 

0  0  jO  '  i0,0  iO, j0'  0 

If  we  were  minimizing  we  would  choose  a  >  0  and  a  would  be 

Jv  U 

non-increasing  . 

For  further  details  on  the  basic  theory  of  linear  programming  see 
any  standard  textbook  such  as,  Hadley,  C.  ,  Linear  Programming, 

Reading,  Mass.,  Addison-Wesley,  1961. 


2.  3  Gomory  Inequalities 


As  mentioned  before,  most  of  our  results  make  use  of  the  Gomory 
Inequalities  (or  Gomory  cuts),  which  were  first  developed  by  Gomory  and 
now  bear  his  name.  In  order  to  make  our  exposition  more  self-contained 
we  will  reproduce  here,  in  a  slightly  modified  form,  the  proof  of  Ben-lsrael 
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and  Charnes  for  the  Gomory  Inequalities  (from  [l  ]). 


If  x  e  L  satisfies  (2.3.  1)  and 


1  hxi 


(2.3.3) 


then 


^  [a./p]  x,  <[aQ/p] 


(2.3.4) 


Proof 


Rewrite  (2.  3.  1)  as 


p  [a./p  1  x.  +  ^  f.  x.  <p  [  aQ/p  ]  +  f( 


(2.3.  5) 


now  there  are  two  possible  cases: 


Either 


f.  x.  «  -  f 
11  0 


(2.  3.  6) 


in  which  case  (2.  3.  4)  always  holds  regardless  whether  x  e  L  or  not,  or 


I  fiV'p 


(2.3.7) 
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in  which  case  we  use  the  following  inequality  which  always  follows  from 
(2.3.  5),  upon  dividing  by  p  ,  and  taking  integer  parts  on  both  sides: 

n  n 


Y  fcj/p]  x4+  (l/p)  ^  fjXi  *  k0/p]  +  f0/p  (2.3.8) 

i=l  i=l 


n 


Now,  since  x  e  L 


•  I  [ai/pi 


x. 

i 


is  an  integer  and,  using  (2.  3.  3), 


i=l 

(2:  3.  7),  and  the  fact  that  OSf  <p,  we  finally  obtain  (2.  3.  4).  This 
proves  part  [  a]  of  Ben-Iorael  and  Charnes  Theorem. 


The  second  part  of  his  theorem  is  actually  a  corollary  of  part  [  a] 
as  we  will  presently  show. 

Part  [  b]  state  8 

Let  p  be  any  positive  real  number,  and  let  g.  be  defined  by 

bi  =  P  <b./p)  -  g.  i  =  0,  1,  . . . ,  n  (2.3.9) 

if  x  e  L  satisfies 


l 


b.  X.  at  b 

11  0 


(2.  3.  10) 


and 


n 

1  hxi-° 
i=l 


(2.  3.11) 
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then 


n 

I  <Vp)  xi  -  <bo/p> 

i=l 

We  state  that  part[  b]  ia  a  corollary  of  part  [  a] 

Proof 

Let  us  define  b.  as 
x 

b.  =  -  a,  i  =  0,  1,  . . . ,  n 

1  i 

We  will  make  use  of  the  fact  that  for  any  real  number 
-  [  c]  =  (-c)  and  [  -c]  =  -  (c) 

We  can  rewrite  (2.  3.  2)  as  follows 

-b.  =  P  [  -b./p]  +  f .  i  =  0,  .  .  .  ,  n 

then,  by  (2.  3.  14)  , 

bi  =  p  <Vp)  -  fi  j 

we  see  that  (2.  3.  9)  is  met  if  we  define 

g.  =  f.  i  =  0,  .  . . ,  n 

x  x 

Relationship  (2.  3.  1)  may  be  rewritten  as 
n 

I  <-bi>  *i  s  -bo 

i=i 
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(2.  3.  12) 


(2.3.13) 
this  holds: 
(2.  3. 14) 


(2.3.15) 


(2.  3.  16) 


(2.  3.  17) 


(2.  3.  18) 


from  which  (2.  3.  10)  follows. 

Also,  (2.  3. 11)  follows  from  (2.  3.  3)  and  (2.  3. 17)  . 

Finally,  (2.  3.  12)  is  obtained  from  (2.  3.  4)  by  rewriting  it  as 

n 

£  t-b/p]  £  t  -b0/p] 

1=1 

and,  by  (2.  3. 14) 
n 

- 1  (Vp>  *i  -  •  (Vp) 

i=l 

We  have  thus  proved  that  part  [  b]  of  the  theorem  is  implied  by 
part  [  a]  .  (The  converse  is  also  true). 

Part  [  a]  of  the  theorem  is  due  to  Gomory  [  6]  but  instead  of 
condition  (2.  3.  3)  he  has  the  stronger  condition  that  x  e  e”  .  He  proves 
the  theorem  for  the  cases  p  =  1  ,  and  p  >  1  . 

Thus,  we  have  proved  that  any  Gomory  cut  generated  from  some 
constraint  8  is  satisfied  by  every  non-negative  integer  vector  x  that 
satisfies  s  .  Whenever  we  have  an  integer  LP  problem  defined  by  a  set 
of  constraints  we  could  add  any  number  of  Gomory  cuts  as  generated  from 
the  original  inequalities,  and  the  expanded  set  of  inequalities  would  still 
be  feasible  for  (satisfied  by)  the  same  set  of  non-negative  integer  vectors. 
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Gomory  Cuts  in  Tableau  Notation 


In  the  tableau,  the 


.th 

1 


row  is 


8. 

1 


s.  >  0 

l 


and  is  equivalent  to 


I 

j=l 


i.J 


t. 

J 


«a.  . 
x,0 


its  Gomory  cuts  will  be: 


1  t,/' 

j=l 

or 


Y  b.  t.  <bA 

L  J  J  0 

j=l 

where 


(2.3.19) 


b.  “  a,  ,/p  J  =  1,  .  • .  ,  n  (2.  3.  20) 

J 


All  the  coefficients  and  variables  of  (2.  3.  19)  are  integers  so  that 
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/ 

/ 

if  we  introduce  a  slack  variable  r  ,  it  will  be  both  non-negative  and  integral 


n 

r  *  b.  +  Y  b.  (-t.)  (2.  3.  21) 

o  L  j  j 

j=l 

The  cut  (2.  3.  19)  or  its  equivalent  (2.  3.  21)  may  be  added  to  the 
original  constraints  by  adding  to  the  tableau  another  row  with  the  coefficient! 

(bo*  br  •  *  *  *  bn>  • 

If  this  added  row  is  used  as  the  pivot  row,  the  pivoting  formulas 
(2.2. 1)  will  be  changed  into: 


(1V 


a'.  =  a 
J  n 


a._  (b./b..) 

}0  /  jO 


j  =  0,...,  n;  j  *  jO  (2.3.22) 


A  Gomory  cut  will  be  called  a  non-zero-cut  if  b_  *  0  and  a  zero- 
*•  - - -  0  — — ■ — 

cut  if  bg  =  0  . 

In  the  next  section  and  in  the  following  chapters  we  will  study 
further  the  theory  and  the  application  of  Gomory  cuts. 


2.4  Development  of  the  Basic  Algorithm 

We  want  to  develop  an  "All-Integer  Primal  Feasible  Integer  LP 
Algorithm".  This  means  that  we  want  the  current  solution  to  always  be 
(primal)  feasible  and  integral;  furthermore,  we  want  all  the  coefficients 
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to  remain  integral. 


If  the  starting  solution  is  feasible  and  all  the  coefficients  are  integer, 
feasibility  can  be  reserved  by  using  the  simplex  criterion  for  pivoting;  and 
integrality  of  the  coefficients  can  be  maintained  by  having  the  pivoting 
elements  equal  to  1  (see  (2.2.1)  ). 

In  general,  it  is  not  always  possible  to  find  pivoting  elements  that 
satisfy  the  simplex  criterion  and  that  have,  at  the  same  time,  a  value  of 
1  .  However,  such  a  pivoting  element  may  always  be  obtained  by  properly 
generating  a  Gomory  cut  and  appending  it  to  the  tableau,  as  will  be  shown 
below. 

Row  iO  is  pivotable  on  column  jO  if  a..  >  0  .  Let  I._  be  the 

r - —  lO.jO  jO 

set  of  rows  that  are  pivotable  on  column  jO  .  A  pivotable  row  iO  is 
most-binding  on  column  jO  if  it  satisfies  the  simplex-like  criterion: 


[a.-  „/a. „  .1  =  min  {fa.  rt/a.  ..]} 

i0f0'  iO,jJ  u  i.O'  i,jOJ' 

16  jo 


(2.4.1) 


If  ckg  <  0  and  if  row  iO  is  most-binding  on  column  jO  ,  we  may 
generate  a  cut  from  row  iO  by  letting  p  =  a  .  This  cut  may  be 

*  J 

appended  to  the  tableau  and  its  element  b  ^  may  be  used  as  the  pivoting 
element.  (Row  iO  will  be  called  the  generating  row,  and  a^  will  be 
the  generating  element).  The  generated  cut  will  not  only  satisfy  (2.4.  1), 
but  it  will  also  satisfy  the  simplex  criterion  (2.2.3).  Furthermore,  the 
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pivoting  element  b.n  will  be  equal  to  1  (as  may  be  checked  from  its 

^  ! 

i 

definition  (2.  3.  20)),  and  the  pivoting  formulas  (2.  3.  20)  may  be  simplified 


a 1  ;  .  n 

JO  JO 


a'.  =  a.  -  a.rt  b. 

J  J  JO  3 


j  =  0, . .  .  ,  n  ;  j  i  jO 


These  new  formulas  show  that  the  integrality  of  the  tableau  elemen 
will  be  preserved  upon  pivoting. 

I 

If  some  row  i  is  pivotable  on  column  j  and  it  has  a.  .  >  a.  it 

a » J  ii  w 

I  l 

will  have  [a^  q/s^  jJ  =  0  and  it  is  clear  that  it  will  be  most-binding  on 
column  j  .  Even  more,  we  say  it  is  zero-binding  because  a  zero-cut  coul 
be  generated  from  it. 

Considering  column  j  ,  if  there  is  at  least  one  pivotable  row  and  if 

none  of  the  pivotable  rows  is  zero-binding  we  have  a  breakthrough  because 

# 

we  can  generate  a  non-zero-cut  (the  breakthrough  is  in  the  maximizing  dii 


tion  if  a.  <  0).  If  there  is  no  pivotable  row  on  column  j  (and  if  a.  <  0)  th< 

1  J  J 

the  prdblem  has  an  unbounded  solution  (which  is  another  type  of  breakthroi 


A  -  If  there  is  a  breakthrough  in  a  negative  column  jO  ,  select  sucl 

j 

a  column  for  pivoting.  If  no  negative  column  has  a  breakthrough  take  som< 
negative  column  jO  for  pivoting.  !  (If  there  are  no  negative  columns  the 
tableau  is  optimal). 

i 

B  -  Having  chosen  jO  ,  take  one  of  the  most-binding  rows  on  colum 


i 
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jO  as  the  generating  row  iO  (if  there  is  none,  the  problem  is  unbounded). 

C  -  From  row  iO  generate  a  Gomory  cut  with  p  =  a^  ^  and  add 
this  constraint  to  the  bottom  of  the  tableau. 

D  -  Pivot  on  element  b..  of  the  added  constraint  and  erase  the 

JO 

constraint  after  having  pivoted.  Go  back  to  A. 

The  basic  algorithm  assumes  that  the  tableau  is  in  primal  feasible 
form  and  that  all  the  coefficients  are  integer.  Also,  it  does  not  specify 
how  to  choose  a  pivoting  column  from  several  proper  candidates,  or  how 
to  choose  a  generating  row  when  there  are  several  most-binding  rows  on 
the  pivoting  column  jO  .  These  matters  will  be  discussed  in  Chapter  4. 

If  we  want  the  algorithm  to  be  minimizing  we  only  have  to  require  that 
the  pivoting  columns  be  positive. 


In  the  process  of  solution  there  will  be  an  improvement  in  the 
solution  vector  whenever  a  breakthrough  is  found  and  used.  Going 
from  one  breakthrough  to  another  constitutes  a  major  iteration.  But  in 
order  to  obtain  a  breakthrough  it  might  be  necessary  to  perform  several 
pivots  with  zero-cuts.  Every  pivot,  whether  it  is  done  with  a  zero-cut 
or  with  a  non-zero-cut,  is  a  minor  iteration  (or  simply,  iteration).  Thus, 
a  major  iteration  requires  a  number  of  minor  iterations. 


2.  5  An  Example 

We  will  solve  here  the  example  given  by  Martin  in  [8] 
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max  z  =  2Xj  +  3x^ 
with  2x^  +  5x^  <  8 

3Xj  +  2x^  <  9 

Introducing  the  slack  variables  and  putting  the  problem  into  the 
tableau  form: 


The  arrows  show  the  column  jO  and  the 

row  iO  .  The  dot  (o.)  marks  the  element 

a..  ..  and  the  asterisk  (*)  marks  the 
iO,jO 

pivoting  element  in  the  added  constraint. 


r  j  =  1  0  1* 

1 


1  1*  -3  r  =  0  -1  1*  r  =  1  1*  -1 

2  t  t 


The  optimal  solution  is  =  3,  x^  =  0,  z  =  6  . 
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CHAPTER  3 


OBTAINING  A  FIRST  FEASIBLE  SOLUTION 


3.  1  Different  Types  of  Variables 

Every  row  of  the  tableau  corresponds  to  an  expression  that  gives 
a  certain  variable  in  terms  of  the  (current)  parametric  variables.  The 
variables  thus  expressed  will  be  called  row  variables  and  they  include  the 
objective,  the  slack  variables,  and  the  problem  variables.  The  constraints 
of  the  original  integer  LP  are  denoted  by  the  fact  that  some  (or  all)  of 
these  row  variables  are  constrained.  Four  types  of  variables  will  be 
studied:  unsigned  variables,  non-negative  variables,  zero-variables  and 
bounded  variables.  In  addition,  the  concept  of  objective  variable  will  be 
discussed. 

3.  1.  1  Unsigned  Variables 

An  unsigned  variable  is  a  variable  that  is  not,  per  se,  necessarily 
constrained  in  sign.  The  most  common  case  of  unsigned  variable  is  the 
one  corresponding  to  the  objective  function,  but  in  some  problems  there 
might  be  additional  unsigned  variables  that  are  pertinent  to  the  formu¬ 
lation  of  the  problem.  The  treatment  of  rows  corresponding  to  unsigned 
variables  is  simple;  they  are  not  considered  as  candidates  for  generating 
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I 

I 


cuts,  but  the  corresponding  coefficients  are  transformed  by  means  of 
the  pivoting  formulas  (2.4.3)  as  are  those  of  any  other  row.  Unsigned 
variables  are  always  considered  to  be  feasible,  of  course. 

3.1.2  Non-negative  Variables 

A  non-negative  variable  may  be  of  any  of  two  types:  a  slack  variabl 
that  was  introduced  to  change  an  inequality  into  an  equality,  or  an  original 
problem  variable  that  was  required  to  be  non-negative.  Non-positive 
variables  are  not  considered  here  because  they  can  be  converted  to  non¬ 
negative  variables  by  a  simple  change  of  variable.  A  non-negative 
variable  s^  is  feasible  if  its  current  value,  a^  ,  is  non-negative.  The 
row  corresponding  to  a  non- negative  variable  will  be  a  candidate  for 
generating  cuts  only  if  it  is  feasible  (for  an  exception  see  section  3.2). 

3.1.3  Zero-Variables  (The  Treatment  of  Equalities) 

One  way  to  deal  with  equalities  in  linear  programming  is  to 
represent  them  by  means  of  opposing  inequalities  (for  instance,  ax  =  b  , 
cx  =  d  may  be  represented  as  ax  <  b  ,  cx  <  d  ,  -(a  +  c)  x  ^  -  (b  +  d)). 

As  far  as  we  know,  this  has  been  the  usual  approach  in  integer  LP  . 

We  have  used  a  different  approach. 


1  See,  for  instance,  Giglio,  R.  J.  and  Wagner,  H.  M.  "Approximate 
Solutions  to  the  Three-Machine  Scheduling  Problem",  Operations 
Research,  v.  12  (1964),  pp.  305  -  324. 
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Take  the  following  equality: 
n 

(3.1.1) 

i=l 

Let  us  introduce  an  artificial  variable  s  and  put  (3.  1. 1)  in  the 
following  form: 

n 

s  =  a0+£  a.  (-x^  (3. 1 . 2) 

i=l 

It  is  obvious  that  any  set  of  x^'  s  that  satisfy  (3. 1.1)  (that  is, 
any  set  of  feasible  x^'  s),  will  make  s  equal  to  zero.  We  call  s  a 
zero-variable  because  it  is  constrained  to  be  zero;  that  is,  s  is 
feasible  only  when  its  value  is  zero.  Furthermore,  a  variable  of  this 
type  is  an  explicit  zero-variable  since  we  explicitly  know  its  nature 
from  the  problem  formulation. 

If,  after  several  pivots,  a  zero-variable  row  has  only  one  non¬ 
zero  element  a.  .  ,  j  +  0  ,  this  means  that  s.  =  a.  .  ( — t . ) »  so  that 
i.J  i  1>)  J 

the  only  feasible  value  of  the  parametric  variable  t.  is  zero.  If  this 
is  the  case  we  might  as  well  make  t  =  0  which  is  equivalent  to  erasing 
the  column  corresponding  to  t^  ,  thus  reducing  the  width  of  the  tableau. 
Furthermore,  if  after  erasing  a  column  there  are  one  or  more  rows 
that  have  only  zero  elements,  (except  perhaps  in  column  zero),  we  may 


I 


a.  x. 

l  l 
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also  erase  these  rows  if  we  take  care  of  storing  somewhere  the  current 
(and  final)  values  of  the  corresponding  row  variables.  The  justification 


for  this  is  that  if  every  element  a.  .  ,  j  #  0  ,  of  row  i  is  equal  to 

zero,  then  there  is  no  way  in  which  a^  ^  could  change  by  pivoting  (we 

would  always  have  a.  ..  =  0  in  the  pivoting  formulas  (2.2. 1)). 

i,jO 

This  reduction  of  the  tableau  is  of  course  equivalent  to  using  the 
equality  in  question  to  solve  for  one  variable  in  terms  of  the  others,  and 
to  substitute  this  expression  in  the  other  constraints  so  as  to  reduce  the 
dimensionality  of  the  problem.  The  algorithm  for  finding  a  first  feasible 
solution  will  do  this  for  us,  automatically. 


But  this  is  not  the  only  way  to  reduce  the  dimensionality  of  the 
problem.  Because  of  the  special  nature  of  integer  LP  problems,  even 
if  there  are  no  equalities  in  the  formulation  of  the  problem,  in  many 
cases  the  dimensionality  of  the  set  of  feasible  integer  solutions  will  be 
smaller  than  the  dimensionality  of  the  space  of  feasible  continuous 
solutions.  Thus,  in  the  process  of  solution  of  a  problem  we  may  find 
out  that  some  of  the  parametric  variables  become,  in  fact,  implicit 
zero- variables.  If  this  situation  occurs  and  we  are  able  to  recognize 
it,  the  tableau  can  be  reduced.  The  following  lemma  gives  the  basic 
rule  for  recognizing  implicit  zero  variables. 

Lemma  3.  1:  If  there  is  a  row  i  such  that  a.  .  >  0  ,  j  =  0  , 
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n  ,  and  this  row  corresponds  to  a  non-negative  variable,  then 

every  column  j  with  a,  .  >  a  corresponds  to  an  implicit  zero- 

x,  j  i,  0 

variable. 

Proof:  Let  J  =  {j|  0  <  a.  .  <  a.  n  ;  j  =  1, . . . ,  n}  and  J+  = 

"  x,  j  l,  0 

{j|  a.  .  >  a.  .  ;  j  :  1 .  n}  ,  Let  us  assume  that  J+  is  not  empty 

i* j  i#  0 

so  that  the  lemma  applies.  Because  row  i  corresponds  to  a  non- 

I 

negative  variable  the  row  represents  the  following  constraint 


/  a.  .  t.  +  V  a.  .  t.  «  a. 
Lt  x,j  j  Lj  .  i.  J  J  x, 

j«  J  j«  J 


A  cut  can  be  generated  by  letting  p  =  min  +  {a.  .}  the  coefficients  of 

j«  J  1,J 

the  cut  will  be: 


b„  =  [a.  _/p]  =  0 
0  1  i, 


b;  =  [a.  ;/p] 


J  x.J 

The  Gomo 


5  1  if  j«  J 
=  0  if  j€  J' 


ry  cut  is  then:  b.  t.  <b_  =  0  with  b.  5  1  for  it  J+  . 

L  *  J  J  0  J 

j«  J 

But  since  “11  the  parametric  variables  t^  can  be  constrained  to  be 

non-negative  (without  losing  feasible  solutions  to  the  original  problem) 

then  the  only  feasible  value  for  the  variables  t.  ,  j«  J  is  zero.  Thus, 

t.  =  0  ,  j«  J*  ,  implies  that  this  set  of  parametric  variables  are  zero- 
J 

variables  and  their  corresponding  column  may  be  erased  in  order  to 
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reduce  the  tableau. 


(Q.E.I 


A  final  remark  about  zero-variables:  a  zero-variable  is  at  the 
same  time  a  non-negative  and  a  non-positive  variable,  and  we  may 
consider  it  so  whenever  it  is  convenient. 


3.1.4  Bounded  Variables 

Many  of  the  practical  problems  formulated  as  integer  LP  problems, 
involve  the  use  of  variables  that  are  constrained  to  be  either  zero  or  one; 
these  variables  are  sometimes  called  Boolean  or  decision  variables. 
Generalizing  this  concept,  let  us  assume  that  the  integer  variable  x.  is 
bounded  as  follows  0  <  x^  ^  (where  M^  is  some  positive  integer). 

This  requirement  could  be  expressed  with  two  separate  rows  of  the  takleat 
one  row  would  express  x^  ^  0  and  the  other  one  would  stand  for 
x?  =  M.  -  x.  5  0  . 

ill 

But  it  is  possible  to  express  both  constraints  with  a  single  row  (this 
saving  computer  memory).  If  we  have  the  constraint 


x. 

l 


a.  .  (-t.)  5  0 
i,  j  j 


we  can  compute  the  counter -constraint 


n 


j=l 


<-\i>  <-y 


^  o 
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Thus,  the  slack  variable  x.  is  feasible  if  M.  -  a.  ^  0  .  When 

1  i  i,0 

x.  is  feasible,  it  will  be  binding  on  column  j  if  a,  .  <  0  ;  whereas  x  , 
l  J  1 

when  feasible,  will  be  binding  on  column  j  if  a  .  >  0  . 

i.) 

In  order  to  handle  a  bounded  variable  it  is  enough  to  state  its  non¬ 
negativity,  the  value  M.  ,  and  the  fact  that  the  variable  ££  bounded  (an 
arbitrary  label  would  be  used  to  specify  this  in  the  computer  program). 
For  analytical  purposes  we  will  think  of  it  as  two  separate  non-negative 
variables. 


3.1.5  Objective  Variables 

Any  variable,  whether  unsigned,  non-negative  or  zero-variable, 
can  be  considered  to  be  the  problem  objective  at  any  stage  of  the  process 
of  solution  of  a  given  problem.  The  motivation  for  doing  this  will  be 
discussed  in  the  next  section. 

If  we  are  taking  s.  as  the  current  objective  function  we  will 
consider  its  row  as  the  zeroth  row,  that  is,  as  the  topmost  row.  (The 
order  of  the  other  rows  should  be  specifically  assigned,  but  the  criterion 
for  assigning  the  order  is  unimportant,  as  long  as  the  rows  retain  their 
ordering).  If  s.  is  being  maximized,  only  lexico-negative  columns  will 
be  considered  proper  candidates  for  pivoting.  If  there  is  no  negative 
column  it  means  that  s.  cannot  be  maximized  further.  (If  s^  is  being 
minimized  only  positive  columns  are  considered  for  pivoting). 
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3.2  Finding  a  First  Feasible  Solution 


Before  proceeding  to  optimize  the  original  objective  function,  we 
require  that  the  tableau  be  in  a  (primal)  feasible  form.  There  are  two 
types  of  infeasibilities  that  we  want  to  remove:  a)  zero-variables  with 
non-zero  values,  b)  non-negative  variables  with  negative  values.  It 
seems  advisable  to  make  the  zero-variables  feasible  first  so  that  we 
may  reduce  the  size  of  the  tableau  as  soon  as  possible. 

One  way  to  deal  with  zero- variables  that  are  not  feasible  is  the 
following:  Let  us  assume  that  s.  is  a  zero-variable  and  its  value  is 
a^  q  >  0  (if  a^  q  <  0  we  could  multiply  the  row  by  -1  and  work  with 
s!  (s|  =  -s.)  which  is  also  a  zero  variable).  We  will  consider  the 
problem  defined  only  by  those  rows  corresponding  to  non-negative 
variables  that  are  already  feasible,  and  we  will  try  to  minimize  s. 
subject  to  s,  5  0  ,  that  is,  considering  row  i  itself  as  a  constraint. 

To  do  this  we  apply  our  basic  algorithm  to  the  subproblem;  that  is,  we 
pivot  in  such  a  way  that  the  non-negativity  of  the  subproblem  variables 
is  preserved  while  we  seek  to  bring  the  value  of  s^  down  to  zero.  Thus, 
we  will  pivot  only  in  columns  that  are  lexico-positive  in  the  subproblem. 
If  there  is  no  such  a  column  for  the  row  i  under  consideration,  and  if 
0  >  0  *  this  means  that  no  feasible  solution  exists  for  the  original 
problem.  (The  solution  space  of  the  original  problem  is  contained  within 
the  solution  space  of  the  subproblem;  hence,  if  the  subproblem  does  not 
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have  any  feasible  solution  neither  does  the  original  problem). 

If  we  want  to  reduce  the  tableau  by  eliminating  the  zero- variable 
we  will  not  be  satisfied  with  driving  its  value  down  to  zero.  We  have  to 
continue  pivoting  in  such  a  way  that  row  i  will  eventually  have  only  one 
non-zero  element.  This  will  be  accomplished  in  a  finite  number  of  steps 
as  will  be  shown  next. 

Lemma  3.2:  If  we  generate  a  Gomcry  cut  from  row  iO  ,  using 

p  =  |  a^  |  #  0  ,  and  we  pivot  on  column  jO  of  the  cut,  the  new  elements 

of  row  iO  will  satisfy  the  following:  a^  jO  =  "  ^  ai0  jO^  ' 

0  <  a'  .<  I  a._  ,  j  *  jO  .  That  is,  every  element  of  row  iO  will  be 

iO, j  1  iO,jO' 

non-negative  and  smaller  than  j  a^  q|  ;  except  the  element  on  column 
jO  ,  which  will  be  negative  and  equal  to  -  |  a^  |  . 

Proof:  We  have  defined  [  c]  to  denote  the  largest  integer  smaller 
than,  or  equal  to  c  .  Thus,  if  b  4-  0  ,  the  following  holds: 

0  « (a/b  -  [  a/b]  )  <  1 

If  b  >  0  we  have 


0  «  (a  -  b  [  a/b  ] )  <  b 


or 


0  <  (a|  b)  <  b 


where 


(a|  b)  =  a  -  b  [  a/b  ] ,  b  >  0 


(3.2.1) 

(3.2.2) 
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This  property  will  be  used  to  prove  the  lemma. 

The  cut  generated  fiom  row  iO  with  p  =  |  a^  will  have  the 

following  coefficients: 


bj  =  ^iO,  j/l  ai0,  jo'  ^ 


i  j  “  0 ,  *  *  §  j  n 


There  may  be  two  cases:  a..  >  0  ,  or  a.rt  <  0  .  If  a.rt  .A  >  0 

1  iO,jO  iO,jO  iO,jO 

the  pivoting  formulas  will  be  those  of  (2.  4.  3),  and  specifically  for 
i  =  iO  ,  we  will  have: 

a  3  =  ‘  ai0,  jO  =  "  I  ai0,  jO^ 

aio,j  =  aio,j  "  aio,  jo  bi  =  aio,  j  "  I  aio,  jo'  taio,/l  aio,  jol  * 


j  =  0, . . . ,  n 


1*  10 


If  a._  <  0  ,  we  will  have  b._  =  -1  ,  and  a..  =  -  I  a._  .  J  .  We  can 

iO.jO  jO  iO.jO  1  i0,j0' 

obtain  the  pivoting  formulas  for  row  iO  from  (2.  3.22),  that  is: 


a*  =  +  a  =  -  a 

iOJO  i0,j0  1  i0,j0' 

a.'  .  =  a. .  .  +  a.rt  ..  b.  =  a..  .  -  I  a.„  ..I  [a.„  ./I  a. „  I  ] 

iO.j  iO,  j  i0,j0  j  i0,j  iO,  jO  1  xO,  y  iO.jO1 


j  =  0,  . . . ,  n 


j  *  JO 


Thus,  in  either  case  we  have  the  same  pivoting  formulas  for  row  iO  , 
and  by  comparing  them  with  (3.2.  1),  we  see  that  Lemma  1  is  indeed 
satisfied.  (Q.E.D. 

Lemma  3.  3:  If  we  have  a  row  i  with  a._  =  0  ,  we  can  reduce  it- 
-  iO 
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in  a  finite  number  of  steps,  so  that  only  one  non-zero  element  will  remain. 

Proof:  The  way  to  obtain  this  reduction  is  to  use  only  positive 
elements  of  row  i  as  generating  elements.  (If  initially  there  are  no 
positive  elements  in  row  i  ,  Wi  may  use  any  of  the  negative  elements  as 
generating  elements.  From  Lemma  3.2  we  see  that  the  other  elements 
of  row  i  will  become  either  positive  or  zero.  If  at  least  one  of  them 
becomes  positive  we  may  apply  the  rule  given  above.  If  they  all  become 
zero  then  Lemma  3.  3  is  already  satisfied.  > 

By  always  taking  positive  generating  elements  from  row  i  ,  we 
are  assured  by  Lemma  3.2  that  they  will  always  be  monotonically 
decreasing.  But  they  cannot  decrease  forever  because  they  are  positive 
and  they  decrease  by  integer  amounts.  Thus,  alter  a  finite  number  of 
iterations  there  will  be  no  more  positive  elements  in  row  i  .  The  only 
way  this  may  happen  is  if  every  element  is  zero  except  the  one  that  was 
used  last  as  generating  element,  as  may  be  checked  from  Lemma  3.2. 

This  proves  Lemma  3.3.  (Q.E.D.) 

The  procedure  for  row  reduction  will  be  accelerated  if  the  smallest 
of  the  positive  elements  is  chosen  each  time  as  the  generating  element. 

When  the  row  of  a  zero-variable  is  reduced  so  that  it  has  only  one 
non-zero  element  a_  ,  j  ^  0  ,  we  know  that  the  parametric  variable 
associated  with  column  j  is  a  zero-variable,  and  we  may  erase  this 
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column,  thus  reducing  the  size  of  the  tableau. 


Once  we  have  eliminated  one  zero-variable  we  take  another  one 
and  repeat  the  procedure  until  there  are  no  infeasible  zero-variables. 
Then  we  may  turn  our  attention  to  removing  the  infeasibility  of  non¬ 
negative  variables,  if  there  is  any  infeasibility  of  this  type. 


We  have  said  that  in  removing  the  infeasibility  of  one  zero- 

i 

variable  the  pivoting  is  done  in  such  a  way  that  the  variables  that  are 
already  feasible,  remain  feasible.  This  would,  perforce,  slowl  down 
the  process  of  removing  that  infeasibility  and  of  reducing  the  tableau. 
Another  strategy  would  be  to  concentrate  :  ’educing  the  elements  of 
the  row  and  in  eliminating  the  zero-variable,  without  worrying  whether 
any  other  variables  become  infeasible.  This  would  bring  about  a  faster 
reduction  of  the  tableau,  and  once  it  has  been  reduced  it  might  b§  more 

efficient  to  worry  about  the  other  infeasibilities.  The  desirability  of 

'  • 

one|!  method  versus  the  other  is,  perhaps,  mostly  a  matter  of  computer 

j  l  1 

programming,  and  we  will  not  discuss  it  further. 


I  In  order  to  remove  the  infeasibility  of  a  non-negative  variable 
s.  (assuming  that  a.  n  <  0)  we  consider  again  the  subproblem  formed 
with  those  rows  that  are  already  feasible,  and  having  as  objective  the 
maximizing  of  s^  .  Only  those  rows  that  are  (lexicographically) 
negative  within  the  subproblem  will  be  considered  for  pivoting.  If, 


at  some  stage,  there  is  no  such  a  column,  and  if  a.  ^  <  0  ,  this  means 
that  cannot  reach  a  non-negative  value  in  the  subproblem,  and 
therefore  there  is  no  feasible  solution  of  the  original  problem.  If  the 


subproblem  is  unbounded  in  some  column  j  (such  that  a^  ^  <  0)  ,  we 

can  then  generate  a  cut  from  row  i  by  letting  p  =  |  a.  .J  .  The  Gomory 

h  j 

cut  will  have  b  <  0  and  b.  <  0  and  after  pivoting  on  b.  ,  a.  will  be 

I  J  h 

non-negative  (this  is  the  type  of  cuts  that  Gomory  uses  in  [6]). 


But  whether  or  not  such  unboundedness  is  found,  the  idea  is  to 
improve  s^  until  it  becomes  non-negative,  and  at  the  same  time  to 
maintain  feasibility  in  the  subproblem.  The  variable  s.  does  not  have  to 
be  optimized  in  the  subproblem;  as  soon  as  it  becomes  non-negative  we 
take  another  infeasible  variable  as  our  new  objective,  if  there  are  any 
left  (in  the  process  of  " satisfying"  a  certain  non-negative  variable,  some 
other  variables  may  also  be  satisfied). 

When  every  variable  is  feasible  we  take  the  original  objective  of 
the  problem  and  we  proceed  to  optimize  it,  preserving  the  feasibility. 

This  technique  for  finding  a  first  feasible  solution  has  been 
incorporated  in  our  algorithm  and  it  has  been  programmed  for  the  IBM  7094. 
Its  convergency  depends,  of  course,  on  the  convergency  of  our  basic 
algorithm. 

The  ideas  behind  this  approach  are  not  exclusively  for  integer 
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linear  programming,  but  they  can  also  be  applied,  with  proper  modifica 
tions,  to  regular  linear  programming. 


CHAPTER  4 


/ 


THE  PROBLEM  OF  FINITE  CONVERGENCE 


4. 1  Considerations  about  Convergence 

At  mentioned  in  Chapter  2,  in  applying  the  basic  algorithm  there 
will  be  major  iterations  (breakthroughs)  and  minor  iterations.  Every 
time  a  major  iteration  occurs,  the  solution  vector  (column  o^)  increases 
lexicographically. 

If  we  are  using  the  maximizing  version  of  the  basic  algorithm,  we 
have  the  following  theorem: 

Theorem  4, 1  If  the  problem  is  bounded  in  every  direction,  there 

can  only  be  a  finite  number  of  major  iterations. 

Proof ^ :  A  sequence  of  major  iterations  will  produce  a  sequence  of 

columns  a^'  s  that  will  be  in  lexico-increasing  order  and  which  will  be 

0 

bounded  above  by  x  ,  the  lexico-largest  feasible  solution: 

0  i  1  i  2  f  10 

“o  <  °o  <  °o  <  •  •  •  <  x 


1  This  proof  follows  to  a  certain  extent  a  similar  proof  given  by  Gomory 
in  [ 6]  . 
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I 


Let  us  assume  that  we  had  an  infinite  sequence  of  this  sort.  The 

vectors  are  all-integer,  so  their  components  charge  only  by  integer 

amounts.  The  first  component  (corresponding  to  the  objective  function) 

cannot  increase  indefinitely  because  it  is  bounded  above  by  z^  ,  the  first 

0 

component  of  x  .  Therefore,  the  first  component  can  only  increase  for 
a  finite  number  of  times  and  it  then  remains  fixed  at  some  value  z'  ^  zq  * 

From  now  on  the  second  component  must  be  non-decreasing,  and 
there  are  two  alternatives:  a)  it  increases  indefinitely,  or  b)  it  increases 
up  to  a  certain  value,  and  then  remains  fixed.  Alternative  a)  is  excluded 
because  we  assume  that  the  problem  is  bounded  in  every  direction,  that 
is,  that  the  feasible  range  of  every  variable  has  an  upper  bound.  If 
alternative  b)  occurs  we  turn  our  attention  to  the  third  component  of 
,  which  offers  the  same  two  alternatives.  In  every  case  alternative 
a)  will  be  excluded,  so  that  if  every  component  increases  up  to  a  certain 
value  and  then  remains  fixed,  the  value  of  every  component  will  be  fixed 
after  a  finite  number  of  major  iterations.  Thus,  the  assumption  of  an 
infinite  sequence  has  been  contradicted  and  the  theorem  is  proved.  (Q.E.I 

However,  if  we  use  the  minimizing  version  of  the  basic  algorithm 
we  have  the  following  theorem,  which  is  stronger  than  the  previous  one: 

Theorem  4.  2  If  the  problem  is  bounded  in  the  direction  of 
minimizing  z  ,  there  can  only  be  a  finite  number  of  major  iterations. 
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Proof:  The  proof  is  similar  to  the  previous  one  except  that  we 

have  now  a  lexi’co-decreasing  sequence  of  solution  vectors,  and  that 

i 

S 

i 

while  z  has  a  lower  bound  by  hypothesis,  the  lower  bound  of  every  other 

[ 

variable  is  automatically  provided,  because  they  all  have  zero  as  a  lower 


bound. 


(Q.E.D.) 


(Any  particular  problem  can,  of  course,  be  solved  by  either  version 
of  the  basic  algorithm,  but  in  view  of  theorems  4.  1  and  4.2  it  appears 
that  the  minimizing  version  is  preferable). 


In  order  to  guarantee  finite  convergence  for  the  overall  algorithm 

i 

it  is  necessary  to  guarantee  that  a  breakthrough  (that  is,  a  major  iteration) 
can  always  be  obtained  after  a  finite  number  of  minor  iterations.  One 
reason  why  a  major  iteration  might  require  a  very  large  or  even  an 
infinite  number  of  iterations,  is  because  at  any  stage  there  may  be  more 
than  one  generating  element  (i.e.  more  than  one  choice  of  iO  ,  jO  ),  and 
if  proper  rules  of  choice  are  not  followed,  some  sort  of  cycling  may  then 
occur. 


The  problem  of  finding  rules  ox  choice  that  will  assure  finite 

convergence  of  the  primal  algorithm  appears  to  be  a  difficult  one.  After 

] 

a  large  number  of  analytical  and  computational  efforts  we  have  not 
succeeded  in  finding  rules  of  choice  that  we  can  prove  will  guarantee 
finite  convergence  in  solving  a  general  problem.  However,  we  report 


i 
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here  two  encouraging  results:  first  a  formal  proof  of  finite  convergent 
of  the  algorithm  when  applied  to  two-dimensional  problems;  and  second, 
the  definition  of  a  family  of  rules  of  choice  (for  the  general  problem) 
which  has  both  analytical  and  computational  appeal. 

4.  2  Finite  Convergence  of  Two-Dimensional  Problems 

We  have  been  able  to  prove  finite  convergence  of  the  algorithm 
for  two-dimensional  integer  LP  problems.  Consider  a  general  two- 
dimensional  integer  LP  problem.  It  can  be  expressed  by  means  of  a 
tableau  with  columns  j  ,  j  =  0,  1 , 2  ;  and  rows  i,  i  =  0, 1 ,  .  .  .  ,  m  .  These 
m  +  1  rows  include  the  objective  function  (row  zero),  and  n  constraints, 
two  of  which  are  the  non-negativity  constraints  of  the  problem  variables. 
The  objective  variable  is  unsigned  and  every  other  row  variable  is  non¬ 
negative.  We  will  assume  that  the  tableau  is  already  in  primal  feasible 
form  since  we  are  concerned  here  only  with  the  basic  algorithm.  Let 
us  say  that  the  objective  function  is  to  be  maximized,  and  let  us  assume 
that  the  problem  is  bounded  in  every  direction.  * 


1  This  does  not  entail  any  loss  of  generality  because  it  can  always  be 
done  by  using  A.  CHARNES  and  W.  W.  COOPER  "regularization" 
method.  See  their  book  Management  Models  and  Industrial 
Applications  of  Linear  Programming.  2  vols.  (New  York:  John 
Wiley  and  Sons,  Inc.  1961)  pp.  42^-427. 
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Types  of  Rows.  Dominance 


m 


One  of  the  characteristics  of  pivoting  on  an  element  of  value  +  1 
(as  it  is  the  case  with  the  basic  algorithm)  is  that  the  pivoting  column 
(c^q)  changes  sign;  and  in  particular,  it  becomes  (lexico-graphically) 
positive. 

Considering  the  two  columns  (a  ,  a  ),  there  are  three  possibil- 

ities; 

a)  Both  are  positive, 

b)  One  is  negative,  the  other  one  is  positive, 

c)  Both  are  negative. 

In  case  a),  optimality  is  already  proved,  so  the  problem  is  solved. 
In  either  case  b)  or  c)  we  can  piv^t  on  a  negative  column  and  it  will 
become  positive,  while  the  other  column  either  becomes  (or  stays) 
positive  or  negative.  If  positive,  optimality  has  been  proved;  if  negative, 


we  are  left  with  a  negative  column  and  a  po 


sitive  column. 


phe  other  one  is  positive 


Whenever  one  column  is  negative  and 
(the  zeroth  column  has  been  excluded  from  these  considerations),  we 
say  that  the  tableau  is  in  normal  form.  We  can  see  that  the  tableau 
will  remain  in  normal  form  until  optimality  is  proved.  (The  pivoting 
column  always  becomes  positive  and  the  other  one  becomes  negative, 
unless  optimality  is  proved).  We  will  denote  by  "a"  the  negati''' 


* 


column  {its  elements  will  be  a.  )  and  by  "b"  the  other  column  (elements 
'  1a 


»ib>- 


A  row  i.,  i  *  0  ,  will  be  classified  according  to  the  signs  of  its 


elements  a.  and  a  : 

la  lb 


Type  a.  a 
1a  lb 


1  0 


7  + 


+ 


Furthermore*  let  us  divide  the  rows  of  type  4  into  two  sub-types. 

S' 

If  aA  a  5  a  a. 
y*  Oa  lb  Ob  la 

4b)  M  an  a. ,  <  a.,  a. 

Oa  lb  Ob  la 


Any  row  of  type  1,  2,  3  or  4a  will  be  called  a  dominated  row.  Any 


row  of  type  4b,  5,  6,  7  or  8  is  a  non-dominated  row.  The  current  level 
is  the  number  of  non-dominated  rows  in  the  current  tableau.  Rows  of 
type  1,  2  or  3  are  called  trivial.  Only  rows  of  type  6,  7  or  8  are 


rivotable  because  thev  are  the  only  ones  with  a.  >  0  . 

-  la 
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Considerations  about  Pivoting 


When  the  tableau  is  in  normal  form  and  a  pivot  is  performed, 
columns  1  and  2  exchange  roles  (if  optimality  is  not  proved).  That  is, 
if  column  1  was  designated  as  column  "a"  before  pivoting,  it  will 
become  column  "b"  after  pivoting;  and  vice  versa.  The  same  is  true 
for  column  2,  of  course. 

The  pivoting  formulas  (2.  4.  3)  may  then  be  expressed  as; 


k+1  k  ,  k  ....  k 

a.„  =  a.„  -  b  _  (i0)  a. 

iO  iO  0  '  la 

k+1 


la 


k  k  k 

a.,  -  b  (iO)  a. 


ib 


la 


(4.2.  1) 


k+1  k 

a  =  -  a. 
ib  la 


where  bl*  (i)  =  [a^  ,/a^  ] 

J  i,  J  i,  a 


i  =  0,  .  .  .  ,  m 


j  =  0,  a,  b;  a. .  >  0 
ij 


(4.2.2) 


The  quantities  b  (i)  will  be  called  b-factors;  they  are  defined 

D  - 

only  for  pivotable  rows.  (The  superscript  k  indicates  that  the  quantity 
corresponds  to  the  k^  tableau). 


Some  Lemmas  for  the  Tableau  in  Normal  Form 

Let  +I^  =  {i|  i  =  1 ,  . . . ,  m  and  a!*  >  0}  . 
a  1  la 

Lemma  4.  1 :  If  after  pivoting  (on  column  a)  on  a  cut  generated 

+  k 

from  iO  (iO  c  I  ),  optimality  is  not  proved,  then  the  following  conditions 

cl 

-f*  k 

hold  for  any  row  i  ,  i«  I  : 

a 
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k  1  k+1 

Condition  A)  If  b,  (i)  >  b,  (iO)  then  b  (i)  =  -  l 
b  D  D 

k  k  k4l 

Condition  B)  If  b,  (i)  =  b,  (iO)  then  either  b  (i)  <  -  1 
b  b  b 

or  row  i  becomes  trivial. 

k  k 

Condition  C)  If  b.  (i)  <  b.  (iO)  then  row  i  becomes  trivial, 
b  d 

Proofs; 

Condition  A:  Let  d  =  bk  (i)  -  bk  (iO),  we  will  then  have  d  J*  1 

D  D 

k  k 

because  b  (i)  >  b,  (iO)  and  the  b>factors  are  integer.. 

D  D 

Applying  (4.  2. 1) 

k+1  k  .  k  ....  k  k  . ' k  ...  k  ,  .  k 

a.  =  a.,  -  b  (iO)  a.  =  a..  -  b.  (i)  a.  +  d  a. 
la  ib  b  la  lb  11a  la 

By  means  of  (3.2.2)  and  (4.2.2) 

k+1  k  k  k 

a  s  (a  |  a  )  +  d  a 
ia  ib  ia  ia 

Therefore,  because  of  (3.2. 1)  and  because  d  >  1  , 

k+1  k 

a.  ?  a.  >  0 

la  la 


Also  a. 


ak  <  0 

ia 


.  k+1,  k+1 

so  that  a.,  <  a. 

'ib  1  la 


,  k+1  r  k+1  .  k+1,  f  |  k+1,  .  k+1, 

then  bb  (.)  =  [»iib/>l(,J  *  t-l  *ib  |/.u  ] 


(Q.E.D.) 


Condition  B;  If  bk  (i)  =<bk  (iO)  we  will  have  by  (4. 2.  2),  (3.2.2) 

D  D 


and  (3.2.1), 
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k+1  .  k  |  k  . 

a.  =  (a  a.  ) 
lb  1  la 


la 


k  k+1 

and  a.  >  a.  5  0 
la  la 

i  k+1  k  ^  n 

also,  a.,  =  -  a.  <0 

lb  la 

k+1 

1)  if  a.  =  0  ,  then  row  i  becomes  of  type  1  ,  that  is,  a 
1a 

trivial  row. 


2)  If  a!*+*  >  0  ,  then  |  a|^| 
la  1  ib  1 


k  k+1 

=  a.  >  a. 
la  la 


therefore 


.k+1...  r  k+1,  k+1,  ,  ,  k+1.  /  k+1,  ,  .  _  _  _  \ 

bb  (l)  =  [aib  /aia  ]  =  H  l/a^  ]  <  "  1  (Q.E.D.) 


ib  '  '  ia 


Condition  C:  If  b!*  (i)  <  b^  (iO)  and  we  let  d  =  b^  (iO)  -  b.^  (i) 

b  b  kb 


then  d  5  1 


now 


but 


k+1  k  k  k  k  ,  k  ...  k  ,  k  .  k  .  k  .  k 
a.  =  a.,  -  b  (iO)  a.  =  a.,  -  b  (l)  a.  -da.  =  (a  a.  )  -  d  a. 
la  ib  b  la  ib  b  la  la  ib'  la  la 


/  k  |  k  .  k  ,  ,  k  k 
(a.,  a.  )  <  a.  and  da.  5=  a. 
ib'  la  la  la  la 


so  that 


k+i  _ 
a.  <0 
la 


also, 


k+1 

aib 


-  a.  <0 
la 


then,  row  i  becomes  of  type  3,  that  is,  a  trivial  row. 


(Q.  E.  D. ) 
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Lemma  4.2:  Every  pivot,  except  perhaps  the  last,  is  done  on  a 

j 

cut  generated  from  a  row  of  type  7. 

Proof:  In  normal  form  the  only  pivotable  rows  are  those  of  type 
6,  7  or  8. 

j 

I 

generated  from  either  a  type  6  or  a  type  8, 
and 


If  we  pivot  on  a  cut 


we  will  have  b!*  (iO)  ^  0 
b 


k+l  k  ,  k  I  k  ^  _ 

-  K  UP)  >  o 


Oa  "Ob  ~b 
(because  by  assumption, 


'  k+l  k 

also,  a„,  =  -  a„  >  0 

Ob  Oa 


Oa 

a^  <  0  and  a^,  5=  0  for  k  ^  1) 
Oa  Ob 


The  solution  would  have  been  proved  optimal  and  the  algorithm 
would  stop.  Therefore,  only  the  last  cut  upon  which  we  pivot  may  come 
from  a  row  of  type  6  or  8;  all  the  other  cuts  must  come  from  rows  of 

» 

type  7.  (Notice  that  optimality  can  also  be  proved  after  pivoting  on  a 
cut  generated  from  a  type  7  row). 

Lemma  4.  3:  Every  dominated  row  stays  dominated,  unless 
optimality  is  proved. 

Proof:  A  dominated  row  may  be  of  type  1,  2,  3  or  4a.  If  row  i 
is  of  type  1,  after  pivoting  (on  a  cut  generated  from  a  type  7  row),  we 

i  • 

will  then  have  i 


I 

l 

k+l  k  ,  k  ....  k  k  _  k+l 

a.  =  a.,  -  b,  (iO)  a.  =  a.,  <  0  ;  a., 
la  lb  b  la  lb 


ib 


k 

a.  =0 
la 
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thus,  it  becomes  a  row  of  type  2,  and  therefore,  it  remains  dominated. 


For  rows  of  type  2,  3  and  4a,  we  may  restate  lemma  4.3  in  the 
following  fashion: 

Y  K 

Lemma  4.  3'  :  Any  row  with  a/  <  0  and  a.,  such  that 
- -  7  la  lb 

lc  1c  k  k 

a..  ^  a„,  a.  will  become,  in  ore  iteration  and  if  optimality  is  not 
Oa  lb  Ob  i  a 

proved,  a  row  of  type  4a. 

This  lemma  states  that  rows  of  types  2  and  3  become  of  type  4a; 
and  those  of  type  4a  remain  of  type  4a.  This  agrees  wi  >  lemma  4.  3. 
Proof  of  lemma  4.  3'  :  If  optimality  is  not  shown  we  shoulu  have 


0  ,  because  we  already  know  that  a^f* 
Oa  Ob 

k+1  k  k  '  k  ....  _ 

a~  =  am.  -  an-  K  <l0)  <  0 


a.  >  0  .  Then, 
Oa 


or 


Oa  Ob  Oa  b 


,  k  /  k  ,  k  .... 

a0b/a0a  bb  (‘0) 


{4.2.  3) 


We  want  to  prove  that  row  i  ,  after  pivoting,  will  be  a  row  of 

type  4a,  that  is,  such  that 

k+1  k+1  ^  k+1  k+1  .  ,  k+1  „  k+1  n 

a.  a.,  $  a.,  a.  ,  with  a.  <  0  ,  a  .  >  0 

Oa  lb  nb  la  la  lb 

Applying  (4.  2.1) 


k+1  k  .  k  ,.n.  k  . 
a0a  =  SCb  -  bb  (l0>  a0a  <  " 

a0h‘  =  -  ao  >  0 
Ob  Oa 


(by  hypothesis) 
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k+1  V  k  .k..-. 

a.  =  a  -  a.  b  (iO 
ta  ib  la  b 

k+1  k 

a  ,  =  -  a  >0 

ib  la 


(4.2.4) 


From  the  original  hypothesis  and  from  (4.2.3) 

.k/k.^.k  /  k.  ,k  .... 

^ib^ia*  *  <a0b/a0a>  >  bb  <l0) 

lc  k  Jq 

remembering  that  a.  <  0  ,  we  have  a.,  <  b,  (iO)  a. 

6  la  xb  b  'Via 

k+1 

and  therefore  a.  <  0  . 

xa 

T  ,  k+1  k+1  k+1  k+1 

Let  us  now  assume  that  a„  a.,  <  a,.,  a. 

0a  xb  Ob  la 

If  we  substitute  the  expressions  (4.2.4)  into  this  inequality,  after 

simplifying  we  obtain 

k  k  k  k 

a  a  <  a  a 
0a  ib  Ob  ia 

which  contradicts  the  original  hypothesis.  Therefore  we  should  have: 

k+1"  k+1  k+1  k+1 

a  a  $  a  a 

0  a  ib  Ob  ia 

k+1  k+1 

We  have  also  proved  that  a.  <0  and  a.,  >  0  ;  so  that  the 

r  ia  ib 

proof  of  lemma  4.  3'  (and  of  lemma  4.  3)  is  now  complete. 

Lemma  4.  4:  A  row  of  type  4b,  in  a  single  iteration,  and  if 
optimality  is  not  proved,  will  either  remain  of  type  4b,  cr  become  or 


type  5  or  8,  but  it  cannot  become  of  type  4a. 

k  k 

Proof:  By  definition,  a  row  of  type  4b  has  a.  <  0  ,  a.,  >  0 

-  ia  ib 

,  k  k  k  k 

and  a.  a  <  a  a.  . 

0a  ib  Ob  ia 


We  will  have 

k+1  k  k  k 

a.  =  a.,  -  b,  (iO)  a. 
la  ib  b  1a 

k+1  k 

a  =  -  a.  >0 
ib  la 

k+1 

We  cannot  say  anything  certain  about  the  sign  of  a.  ;  it  may  be  positive, 

1  2L 

zero,  or  negative.  If  it  is  positive,  row  i  has  become  of  type  8;  if  it  is 
zero,  row  i  has  become  of  type  5;  if  it  is  negative,  row  i  has  remained 
of  type  4. 

k+1 

Let  us  assume  that  a.^  <  0  ,  and  that  row  i  has  become  of  type 

a  .  k+1  k+1  k+1  k+1  ...  ,  , 

+a,  that  is,  that  an  a.,  5  a.,  a.  .  Putting  this  in  terms  of  elements 

0a  ib  Ob  ia 

of  the  k**1  tableau  and  simplifying  we  get 

k  k  k  k 

a  a  5  a  a 
0  a  ib  Ob  ia 

but  this  contradicts  the  original  hypothesis;  so  that  we  should  have 
k+1  k+1  k+1  k+1  ,  ,  . 

a0a  aib  <  a0b  &ia  '  an<*  thls  ?roves  mat  row  *  may  remain  of  type 
4b,  but  cannot  become  of  type  4a. 


Lemma  4.  5:  A  row  of  type  5  will  become,  in  a  single  iteration  and 


if  optimaiuv  is  no.,  shuw..,  a  row  of  type  6. 

Proof;  If  row  i  is  of  type  5,  then  by  definition 


k  k 

a.  =  0  ,  a  ,  >  0 
la  ib 


after  pivoting: 
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k+1  k  .  k  ....  k  k  , 

a.  =  a  ,  -  b.  (iO)  a.  =  a..  >  0 
1a  lb  b  la  lb 

k+1  k 

a..  =  -  a.  =0 

lb  1a 


Thus,  row  i  has  become  of  type  6. 


Lemma  4.  6:  A  row  i  of  type  6  or  8 


will  become,  in  a  single 


iteration  and  if  optimality  is  not  proved,  a  fow  of  type  7.  Furthermore, 
k+1 

it  will  have  b  (i)  =  -  1 
b 


Proof:  By 


definition,  if  row  i  is  of  type  6  or  8,  it  will  have 


k  *  k 

a  >  0  ,  a  ,  5  0s  . 
la  ib 

If  optimality  is  not  proved  this  means  that  we  pivoted  on  a  cut 

generated  from  a  row  of  type  7  (lemma  4.2).  Therefore,  we  will  have 

b*  (iO)  <  -  1  . 
b 

Then 

» 

k+1  k  k  ....  k  k 

a.  =  a.,  -  b  (iO)  a.  £  a.  >0 

la  ib  b  la  la 

k+1  k  i  ^  n 

ib  iai  , 


Thus,  row  i  is  now  of  type  7. 


Also, 


.k+1  ...  r  k+1  ,  k+1  i  ,  ,  a  ^  t  k+l  /  k+1. 

bb  {i)  =  laib  /aia  1  =  ‘  1  because  I  1  (aib  /aia  )<0  * 


Lemma  4.  7:  A  row  i  of  type  7  may  become,  in  a  single  iteration 
and  if  optimality  is  not  proved,  a  row  of  type  1  or  3,  or  it  may  remain  of 
type  7. 


I 


Proof:  By  definition,  if  row  i  is  of  type  7  it  will  have 


ak  >  0  ,  ak  <  0  .  Then, 
i  a  lb 

k+1  k  k  .  .  k 

*ia  ’  aib  '  bb  (l0) 


ia 


k+1  k 

a..  =  -  a.  <  0 

lb  la 

k  k+1 

If  optimality  is  not  proved  this  means  that  b^  (iO)  <  -  1  .  Now,  a^ 

k+ 1 

is  necessarily  negative,  but  a.  is  the  difference  of  two  negative 

X  cl 

quantities;  therefore,  it  may  either  be  positive,  negative,  or  zero.  Row 
i  will  then  become  of  type  7,  3  or  1  resepctively. 


The  Graph  of  Changes  of  Type 

From  lemmas  4.  3  to  4.  7  we  realize  that  a  row  may  change  its  type 
in  a  single  iteration  (and  if  optimality  is  not  shown)  but  only  to  certain 
other  types.  The  possible  changes  can  be  summarized  with  the  following 
graph: 


Non-dominated  rows  Dominated  rows 
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It  will  be  noticed  that  there  it?  a  possible  flow  of  non-dominated 
rows,  but  not  vice  versa;  that  is,  we  have  proved  that  the  level  is  non¬ 
increasing. 

A  Theorem  on  Convergence  of  Two-Dimensional  Problems 

Lemma  4.  8:  Unless  optimality  is  proved,  or  unless  a  breakthrough 
is  found,  the  level  can  be  decreased  by  at  least  one  in  a  finite  number  of 

iterations.  - . -  - - -  - - ' .  . . .  .  .. 

til 

Proof:  If  there  were  no  pivotable  rows  in  the  k  tableau,  the 
problem  would  have  an  unbounded  solution  and  this  would  contradict  our 
initial  assumption. 

If  there  are  only  pivotable  rows  of  type  6  or  8,  but  none  of  type  7, 
then  at  least  one  of  them  is  most-binding  and  the  solution  will  be  proved 
optimal  as  was  shown  in  lemma  4.2. 

But  let  us  assume,  for  the  sake  of  argument,  that  there  is  at  least 
one  row  of  type  7,  and  that  no  row  of  type  6  or  8  does  ever  become  most  ■ 

binding.  Of  the  set  of  rows  of  type  7  there  will  be  some  rows  i  with  ! 

. . . ...  ...  ...  _  _ _ _  _  _ _ _ _ _ _ _ _ _  .  .  .  . 

bb  (i)  =  -  1  and  some  with  bfe  (i)  <  -  1  .  Let  us  call  the  two  subsets 

7k  .  7k 

Ij  an«  I2  *  respectively.  There  will  be  two  cases: 

_  ,  7k  ,  ,7k 

Case  I:  I^  empty  (and  1^  not  empty) 

7  k  7  k 

Case  II:  I^  not  empty  (  1^  either  empty  of  not  empty) 

In  case  I,  after  one  iteration  and  if  optimality  is  not  proved,  we 
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will  have  case  II;  because  by  condition  B,  lemma  4.  1,  every  row  in  I 


7_k 
1 


7  k+1 

will  be  in  T  ,  (l  -  r  '  rm  tiling  this  iteration,  one  or  more  of  the  rows 

iU  thay  become  trivial,  in  which  case  the  lemma  is  also  satisfied.) 

7_k 


■  In  case  II,  if  there  is  any  most-binding  row  in  1^  and  we  pivot  on 


the 


7_k 


cut  generated  from  it,  then  every  row  of  I^  will  become  trivial, 


because  b  (i 0)  =  -  1  and  because  of  condition  C  of  lemma  4.  1;  and  the 
b 

level  would  therefore  decrease,  thus  satisfying  the  lemma. 

7  k 

So,  let  us  assume  that  at  least  one  row  in  I^  is  most-binding  and 

that  we  pivot  on  the  cut  generated  from  it  (and  optimality  is  not  proved 

7k  7  k+1 

upon  pivoting).  Every  row  in  1^  will  be  in  1^  as  condition  A  of 

k  lc 

lemma  4.  1  shows  (because  b,  (iO)  <  -  1  and  b,  (i)  =  -  1  for  every 

b  b 

7  k  7  k+i 

i«  1^  ).  Any  rows  that  were  of  type  6  or  8  will  come  to  belong  to  1^ 

7  k 

as  lemma  4.6  shows.  Furthermore,  any  row  of  I^  which  had 

k  k  7  k+1 

b  (i)  >  b,  (iO)  will  become  of  the  set  I.  because  of  condition  A  of 
b  b  1 

7  k  k  k 

lemma  4.  1.  If  there  were  any  rows  in  I_  such  that  b  (i)  <  b  (iO)  , 

6  b  b 

then,  By  condition  C  of  lemma  4.  1,  they  would  become  trivial  and  the 
lemma  would  be  satisfied. 

7  k+1 

The  only  rows  that  would  belong  to  I^  would  have  belonged  to 

7  k  k  k 

I,  and  would  have  had  b  (i)  =  b  (iO).  The  only  way  the  level  could 
c  b  b 

remain  unchanged  is  if  we  could  always  be  able  to  find  a  most-binding 
7  h 

row  in  the  sets  I^  ,  h  =  k  ,  k+1,  .  .  .  ,  and  if  none  of  the  rows  in  these 
sets  became  trivial. 


-57- 


7  H  It  k 

However,  any  row  i  in  would  have  had  (i)  =  (iO)  , 

k+1  k+1  h  h 

b*  (i)  =  b*  (iO),  ....  b,  (i)  =  b,  (iO)  ,  and  therefore,  because  of 
b  b  b  b 

lemma  3.  2  and  {4,  2.1)  we  will  have 

k  k+1  ^  h-1  ^  h 

a.  >  a.  >...>'  a.  >  a.  5  0 
la  la  la  la 

Clearly,  because  the  a.  1  s  are  integer  and  a.  is  finite,  there 
7  la  ia 

7  h 

will  be  a  tableau  h  in  which  every  row  i  ,  ic  »  would  be  such  that 

h  h  ^  _ 

a.n  >  a.  5  0 
iO  la 

7  k 

This  means  that:  either  at  least  one  of  the  rows  originally  in 

became  trivial  at  some  tableau  h  (when  a!1  =  0);  and/or  a  tableau  h 

la 

7  h 

has  been  found  in  which  no  row  of  is  zero-binding.  Therefore,  if 

7  h 

there  is  any  zero-binding  row  in  1^  we  are  forced  to  pivot  on  the  cut 

7  h 

generated  from  it  and  thus  every  row  in  would  become  trivial  and 

7  h 

the  lemma  would  be  proved.  (The  set  would  not  be  empty  because 

by  condition  B  of  lemma  4. 1 ,  it  would  contain  at  least  the  row  from  which 

the  last  cut  w?3  wmerated;  ”nless  that  row  has  become  trivial,  in 

which  case  the  lemma  is  also  satisfied.  )  Otherwise,  if  there  is  no 

7  h 

zero-binding  row  in  1^  either,  this  means  that  a  breakthrough  has 


been  found,  and  the  lemma  is  again  satisfied. 


(Q.E.D.) 


Theorem  4.  3:  If  the  two-dimensional  problem  is  bounded  (as  in 
lemma  4.8)  then,  in  a  finite  number  of  minor  iterations  (pivots),  an 
optimal  solution  will  be  found  and  its  optimality  will  be  proved. 
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Proof:  Any  tableau  in  normal  form  has  a  finite  number  of  non 


dominated  rows  (i.e.  the  level  is  finite);  therefore,  by  repetitive 
application  of  lemma  4,  8  we  see  that,  unless  optimality  is  proved  or  a 
breakthrough  is  found,  the  level  will  decrease,  and  it  could  become  zero 
in  a  finite  number  of  iterations.  But  this  would  imply  that  the  problem  is 
unbounded  and  this  is  contrary  to  our  assumptions.  Therefore,  in  a 
finite  number  of  iterations  either  a  breakthrough  is  found  or  optimality  is 
proved.  This,  together  v/ith  theorem  4.  1  proves  the  present  theorem. 

(Q.E.P.  ) 

It  should  be  pointed  out  that  neither  finding  a  breakthrough,  nor 
pivoting  after  it  has  been  found,  does  necessarily  imply  a  decrease  in  the 
level. 

Comments  on  the  Proof  of  Finite  Convergence 

It  should  be  noticed  that  the  proof,  besides  boundedness  of  the 
problem,  only  requires  that  the  tab'cau  be  in  normal  form;  and  this 
can  be  accomplished  by  using  at  most  one  iteration;  but  it  does  nor 
require  the  use  of  any  specific  rule  for  choosing  one  among  the  several 
most-binding  rows  that  might  be  available  for  pivoting,  and  it  does  not 
require  that  rows  of  type  6  or  8  (which  imply  immediate  optimality)  be 
recognized. 

However,  the  performance  of  the  algorithm  with  a  two-dimensional 
problem  can  be  improved  greatly  if  the  following  rule  is  used. 
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Rule:  Among  the  available  most-binding  rows  (in  case  there  are 
ties)  choose  iO  such  that 

b£  (iO)  =  max  {b£  (i)} 

*  i«  I 

where  I  is  the  set  of  most-binding  rows. 

This  rule  has  two  convenient  effects:  a)  it  accelerates  the  decreasi 

ft 

of  the  level,  b)  it  "recognizes"  and  gives  preference  to  rows  of  type  6  or 
8,  thus  shortening  the  process. 

4.  3  A  Promising  Family  of  Rules  of  Choice  for  n-Dimensional  Convergent 

Among  the  many  rules  of  choice  that  were  tried  analytically  and 
computationally  for  convergence  of  general  problems,  one  class  of  rules 
finally  emerged  as  the  most  reasonable,  and  it  has  indeed  given  encour¬ 
aging  computational  results.  We  have  called  this  family  of  rules: 
hierarchical  rules  of  choice.  Let  us  consider  the  n  current  k-dimension 
problems  P(k)  ,  constituted  by  the  first  k  current  parametric  variables 
(1  <  k  ^  n)  plus  the  zero^  column.  This  establishes  a  current  hierarchy 
of  problems;  that  is,  P(k)  is  a  sub-problem  of  F(k  +  1)  ,  which  in  turn 
is  a  sub-problem  of  P(k  +  2)  ,  and  so  forth.  P(n)  is,  of  course,  the 
original  problem. 

The  rules  will  be  presented  by  giving  the  algorithm. 
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Hierarchical  Algorithm 


A)  If  there  is  no  breakthrough  on  any  column  go  ->  C,  otherwise  go 


to  B. 


B)  From  the  columns  with  a  breakthrough,  select  the  most  negative 
one  as  the  pivoting  column.  Choose  one  of  the  most-binding  rows  as 
generating  column,  generate  the  Gomory  cut,  pivot  and  go  back  to  A. 

C)  Select  the  smallest  not  lexico-optimal  problem  F(k).  That  is, 

f  | 

k  is  chosen  so  that  <  0  while  <v  >  0  ,  j  =  l,...,k-j.  (This 
implies  that  the  current  P(k-l)  has  been  proved  lexico-optimal.  )  If 
there  are  no  negative  columns  the  lexico-optimal  solution  has  been  found 
and  its  lexico-optimality  has  been  proved. 

D)  In  order  to  solve  the  current  T'\k)  We  O’ tract  from  it  a  (k  -  1) 
dimensional  not  lexico-optimal  *ub-problem.  Thy*  means  that  we  have 

to  reorder,  in  some  specific  way,  the  first  k  ooiumns  » |i  that  the  negative 
column  will  now  occupy  one  of  the  fi  i  st  V.  t  yla  .  ft  (that  is,  without 
counting  column  zero).  This  new  fisst  k  -  1  column  will  define  our  new 
P(k  -  1).  This  reordering  is  done  recursively  so  that  a  new  hierarchy 
of  sub-problems  P(j)  ,  j  =  1,...,  k  -  1  is  established.  The  negative 
column  (previously  the  k  )  will  now  occupy  the  first  position  and, 
together  with  ,  constitutes  the  new  P(l).  The  first  column  is  now 
chosen  as  the  pivoting  one. 

E)  Considering  again  the  complete  tableau,  one  of  the  rows  that 
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is  most-binding  on  column  1  is  selected  as  the  generating  row,  a  cut  is 
generated  from  it,  and  after  pivoting  (thus  possibly  modifying  the  whole 
tableau)  one  goes  back  to  A. 

In  step  D  we  implicitly  extract  from  P(k)  ,  by  reordering,  a  new 

P(k  -  1)  because  the  current  one  has  been  proved  lexico-optimal.  It  is 

in  this  re -extracting  cf  a  sub -problem  that  the  main  theoretical  difficulty 

lies,  because  in  order  to  assure  convergence,  this  new  (k  -  1)- 

dimensional  sub -problem  has  to  "dominate",  in  some  fashion,  the  current 

P(k  -  1)  whose  lexico-optimality  has  just  been  established.  In  other 

words,  if  the  current  P(k  -  1)  has  been  proved  lexico-optimal,  it  means 

that  no  better  solution  for  P(n)  is  found  in  the  (k  -  l)~dimensional 

polyhedron  defined  by  t.  =  0  ,  j  =  k,  .  .  .  ,  n  .  Thus,  we  have  to  change 

one  of  those  parametric  variables  specifically,  by  reordering  the  first 

k  columns  as  described  above,  we  bring  in  a  new  t,  .  This  new  t, 

k  k 

would  have  to  be  more  "dominating"  than  the  old  one.  Although  this 
dominance  has  been  established  for  2 -dimensional  problems,  it  has  not 
been  generalized  to  larger  problems.  The  way  we  have  used  to  go  around 
this  difficulty  is  described  next. 

Some  Heuristic  Members  of  the  Family 

Since  no  analytical  grounds  for  extracting  the  sub-problems  have 
been  discovered,  we  have  resorted  to  heuristics  (and,  anyway,  heuristics 
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are  usually  the  roads  to  analytical  grounds). 

If  the  current  P{k  -  1)  is  lexico-optimal  and  a  <  0  ,  extracting 

IV 

a  new  P(k  -  1)  from  P(k)  is  equivalent  to  upgrading  one  of  the  first 
(k  -  1)  columns  to  the  k  ^  place  and  making  a  one  of  the  first  (k  -  1) 
columns.  It  is  in  selecting  that  column  of  P(k  -  1)  which  is  to  be 
upgraded,  that  a  heuristic  can  be  introduced.  Some  possible  heuristics 
are:  a)  upgrade  the  (k  -  l)1^  column  (heuristic  JA),  b)  upgrade  the 
first  column  (heuristic  JB),  c)  upgrade  the  column  upon  which  we  have 
pivoted  most  recently  (heuristic  JC),  d)  upgrade  the  lexico-largest 
column  (heuristic  JD). 

th 

Rule  A  when  applied  recursively,  would  put  the  k  column  in 
the  first  place  and  would  shift  the  first  (k  -  1)  columns  one  place  up. 
Rule  B  would  reverse  the  order  of  the  first  k  columns.  ’'Rule  D 
would  rearrange  the  first  k  columns  in  lexico-decreasing  order.  In 
any  case,  the  pivoting  column  (which  previously  was  the  k**1)  would  end 
up  in  the  first  position. 

Once  the  pivoting  column  has  been  chosen,  and  the  columns  have 
been  reordered  it  is  necessary  to  choose  a  pivoting  row  among  the  ones 
that  are  most-binding  on  the  pivoting  column.  The  following  are  two  of 
the  possible  heuristic  rules  of  choice:  a)  take  the  first  most-binding 
row  (heuristic  IA),  b)  take  the  most-binding  row  that  generates  the 
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lexico-largest  Gomory  cut  (heuristic  IB).  Heuristic  IB  is  a  direct 
extension  of  the  rule  given  at  the  end  of  section  4.2.  (A  row  vector  B^ 
is  lexicographically  larger  than  another  row  vector  B^  if  the  first 
element  where  they  differ,  counting  from  left  to  right,  is  larger  in 
than  in  B .  . ) 

J 

4.  4  Computational  Experience 

We  have  not  tried  out  every  combination  of  heuristics  nor  even 
every  heuristic.  Only  the  following  three  combinations  have  been  studied 
JA  -  IA,  JA  -  IB,  and  JB  -  IB.  Combinations  JA  -  IA  and  JA  -  IB  were 
successful  with  many  problems  but  JA  -  IB  was  more  efficient:  JA  -  IB 
solved  problems  that  were  not  solved  (in  a  reasonable  amount  of  time)  by 
JA  -  IB,  and  those  problems  that  were  solved  by  both  combinations  were 
solved  faster  by  JA  -  IB. 

A  few  small  problems  were  tried  with  combination  JB  -  IB  but 
none  were  solved.  Thus,  we  have  selected  combination  JA  -  IB  as  a 
good  heuristic. 

In  order  to  compare  the  performance  of  our  algorithm  with  those 
of  other  authors  we  have  selected  the  nine  examples  found  in  ref.  [  10]. 
Thompson  has  used  those  examples  to  compare  his  method  with  the  all¬ 
integer  method  of  Gomory  [6].  The  results  are  shown  in  Table  4.1. 
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Jn  analysing  these  results  it  should  be  considered  that  the  iterations  of 
Gomory' s  all-integer  algorithm  are  computationally  equivalent  to  ours. 
Thompson  reports  two  figures  for  performance:  the  number  of  stopped 
linear  programming  problems  that  were  solved  (L.  P. ' s),  and  the  total 
number  of  pivots  required  for  their  solution  (last  column  of  Table  4.  1). 
He  also  states  that  the  solution  of  one  of  the  stopped  LP  problems  takes 
about  the  same  as  one  of  Gomory'  s  iterations,  whether  it  is  done  by 
hand  or  by  machine. 

*  1.  li 1 

The  nine  problems  are  initially  in  dual  feasible  form  which  would 
seem  to  be  advantageous  to  both  Gomory' s  and  Thompson' s  algorithms. 

Our  algorithm  was  programmed  in  such  a  way  that  whenever 
several  columns  had  a  breakthrough,  the  first  of  these  columns  was 
chosen,  rather  than  the  most  negative.  # 

i 
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TABLE  4.  1  NINE  EXAMPLES  SOLVED  WITH  DIFFERENT  METHODS 


Example 

Dimensions 

Rule  JA  -  IB 

Optimal  Found 

of  Tableau 

Feasible 

Optimal 

Optimal 

GOMORY 

THOM  PS  C 

F  ound 

F  ound 

Proved 

(iterations) 

L.  P. ' s  Pi' 

(I 

eration  Nc 

•) 

1 

5x3 

5 

7 

8 

13 

20 

2 

7  x  4 

35 

37 

37 

1000* 

? 

3 

7x4 

24 

25 

84 

1570 

■ 

173  1 

4 

11x6 

500* 

- 

- 

? 

? 

5 

5x3 

51 

102 

102 

? 

255  1 

6 

5x3 

102 

102 

202 

? 

260  1 

7 

5x3 

102 

102 

202 

? 

255  1 

8 

17  x  9 

4 

12 

74 

5215 

13 

9 

18  x  8 

1000* 

- 

- 

? 

2038  108 

Other  Problems  Tried 


A 

8x6 

0 

17 

17 

B 

20  x  13 

14 

19 

20 

C 

25  x  13 

78 

- 

300* 

D 

43  x  19 

16 

22 

38 

The  asterisk  (*)  indicates  that  the  program  was  stopped  after  this  many 
iterations  without  having  found  the  answer. 

^  ^This  is  the  total  number  of  iterations  required  to  solve  the  problem 
(it  includes  the  iterations  used  for  finding  a  first  feasible  solution).  A 
question  mark  (?)  indicates  that  the  figure  was  not  reported  in  ref.  [10]. 
It  does  not  imply  that  the  problem  was  not  solved. 
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CHAPTER  5 


INTEGER  L.  P.  AND  NUMBER  THEORY 

In  the  present  chapter  we  will  develop  a  generalization  of  the 
Euclidean  Algorithm,  and  we  will  explore  the  application  of  integer  L.  P. 
to  the  solution  of  linear  diophantine  equations.  The  importance  of  these 
problems  and  of  their  solution  is  well  established  in  number  theory  (for 
instance,  see  refs.  [23]  and  [24]). 

5.  1  Generalized  Euclidean  Algorithms 

Whenever  we  talk  of  the  greatest  common  divisor  (g.  c.  d.  )  of  two  or 
more  numbers,  or  of  a  vector  of  numbers,  it  will  be  with  the  implicit 
understanding  that  the  numbers  are  integer,  and  that  we  are  referring  to 
the  g.  c.  d.  of  their  absolute  value. 

The  Euclidean  Algorithm  finds  the  g.  c.  d.  of  two  numbers  by 
performing  successive  divisions  (where  the  divisor  is  the  previous 
remainder  and  the  dividend  is  the  previous  divisor),  until  the  remainder 
of  a  division  becomes  zero.  The  absolute  value  of  the  last  divisor,  which 
is  also  the  last  non-zero  remainder,  is  the  g.  c.  d.  of  the  two  original 
numbers.  The  procedure  terminates  in  a  finite  number  of  steps. 
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The  traditional  way  of  finding  the  g.  c.  d.  of  more  than  two  numbers 
has  been  to  find  first,  by  means  of  the  Euclidean  Algorithm,  the  g.  c.  d.  of 
two  of  the  numbers;  say  it  is  d^  .  Then  the  g.  c.  d.  of  d^  and  another  one 
of  the  numbers  is  found,  and  so  forth,  until  all  of  the  numbers  have  been 
taken  into  consideration.  The  final  number  obtained  is  shown  to  be  the 
g.  c.d.  of  all  the  original  numbers  taken  together. 


We  propose  here  a  more  general  procedure,  which  is  based  in  what  1 
we  call  elementary  operations  on  a  vector.  Consider  a  vector 

t- 

A  =  (a  ,  .  .  .  ,  a  ).  An  elementary  operation  is  either:  changing  the  sign 

of  one  element,  or,  adding  or  subtracting  from  element  a.  an  integer 

multiple  of  another  element  a.  ,  j  ^  i  .  Thus,  a.  will  be  called  the 

J  V 

element  to  be  transformed,  and  a.  the  transforming  element. 


Lemma  5.  1:  The  g.  c.  d.  of  a  vector  is  unchanged  by  elementary  t 


operations. 

j. 

Proof:  Let  A  =  (a  ,  a  ,  .  .  .  ,  a  ),  where  the  a.'  s  are  integer  and  i 

1  fa  n  i  I. 

i 

at  least  one  of  them  is  different  from  zero.  Let  d  be  the  g.  c.  d.  of  A. 

(If  there  are  ar.y  zero  elements  this  does  not  make  any  difference  because 
any  number  is  the  divisor  of  zero.  Thus,  the  g.  c.  d.  of  vector  A  as  a 
whole  is  the  same  as  the  g.  c.  d.  of  the  non-zero  elements  of  A). 

One  of  the  two  types  of  elementary  operations  is  to  change  the  sign 
of  one  of  the  elements;  this  will  obviously  not  affect  the  g.  c.  d.  of  the 
vector. 
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The  other  type  of  elementary  operation  is  where  a.  is  changed 

into  a.  +  k  a.  ;  i  *  j  ,  k  =  0,  t  1,  t  2.  ...  Let  us  call  A'  the  vector  A 

i  J 

after  we  have  applied  one  elementary  operation  of  the  last  type.  We  want 
to  show  that  the  g.  c.  d.  of  A'  is  the  g.  c.  d.  of  A  .  Say  that  d  is  the 
g.  c.d.  of  A  .  We  have  to  show  that  d  is  a  common  divisor  of  A'  ,  and 
that  there  is  no  larger  common  divisor  of  A'  .  If  d  is  the  g.  c.d.  of  A 
we  can  put 

A  =  (a. ....  ,  a  )  =  (db  ,  db  ,  . . .  db  ) 

I  n  l  c  n 

After  the  elementary  operation  the  only  element  that  will  be  changed  will 
be  a!  .  We  will  have 

l 

a!  =  a.  +  ka.  =  db  +  kdb.  =  d(b.  +  kb.) 

l  l  j  l  j  i  j 

Thus,  d  will  be  a  common  divisor  of  A'  .  Let  A^  be  the  (n  -  l)-element 

vector  that  remains  after  we  remove  a.  from  A  .  The  g.  c.  d.  of  A.  is 

x  1 

either  d  or  a  multiple  of  d  .  If  it  is  d  ,  then  the  g.  c.  d.  of  A'  , 

A'  =  (A.,  a!)  =  (A  ,  d(b.  +  kb.)),  is  also  d  because  of  the  associative  law 

ii  i  i  j 

of  the  g.  c.d.  (see  ref.  [23],  p.  48). 

On  the  other  hand,  let  us  assume  that  the  g.  c.  d.  of  A.  is  cd  , 

l 

where  c  is  a  positive  integer  larger  than  one.  The  only  way  in  which 

the  g.  c.  d.  of  A'  would  be  larger  than  d  ,  would  be  if  the  g.  c.  d.  of  cd 

and  a!  =  d(b.  +  kb.),  was  larger  than  d  ;  and  this  could  only  occur  if  c 
^  ^  J 

and  bi  had  some  common  divisor,  as  d^  ,  (d^l).  But  if  this  were  true. 
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then  the  g.  c.  d.  of  A  would  have  been  d^  in  the  first  place.  It  follows 
that  c  and  b.  do  no  have  any  common  divisor,  and  therefore,  d  is  the 
g.  c.  d.  of  A'  ,  which  proves  the  lemma.  (Q.  E.  D.  ) 

We  can  now  propose  the  following  theorem  that  justifies  the 
Generalized  Euclidean  Algorithm  (G.  E.  A.  ). 

Theorem  5.1:  If  we  have  an  n-dimensional  vector  and  we  perform  a 
sequence  of  elementary  operations  on  it,  so  that  only  one  non-zero  element 
remains;  then  the  absolute  value  of  this  last  number  is  the  g.  c.  d.  of  the 
original  vector. 

Proof:  The  g.  c.  d.  of  a  vector  that  has  only  one  non-zero  element 
is  precisely  the  absolute  value  of  that  element.  Therefore,  by  lemma  5.  1 
it  follows  that  this  is  the  g.  c.  d.  of  the  original  vector.  (Q.  E.  D'.  ) 

Any  rule  that  guarantees  the  "reduction"  of  the  original  vector  (so 
that  it  will  have  only  one  non-zero  element)  in  a  finite  number  of  steps, 
will  constitute  a  valid  G.  E.A.  The  following  rule  provides  a  family  of 
convergent  G.  E.  A. ' s. 

Rule:  Always  apply  elementary  operations  such  that  the  absolute 
value  of  the  transformed  element  is  smaller  than  its  previous  absolute 
value,  and  smaller  than  the  absolute  value  of  the  transforming  element. 

The  Euclidean  Algorithm  is  a  particular  case  of  the  G.  E.  A.  because 
finding  the  remainder  of  the  division  is  an  elementary  operation  that 
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satisfies  the  previous  rule. 


These  results,  although  they  are  somewhat  stronger  and  have  been 
developed  independently,  are  similar  to  those  of  W.  A.  Blankinship 
(ref.  [22]).  He  not  only  finds  the  g.c.d.  of  n  positive  integers,  but  also 
finds  one  solution  to  the  diophantine  linear  equation  that  has  these  n 
integers  as  coefficients  and  the  corresponding  g.  c.  d.  as  constant  term. 
His  approach  does  not  explicitly  involve  integer  L.  P. 

In  the  following  sections  we  will  develop,  by  means  of  integer  L.  P.  , 
much  more  general  results  for  the  solution  of  systems  of  diophantine 
equations. 

5.  2  General  Solution  of  Linear  Diophantine  Equations 

A  linear  diophantine  equation  is  a  linear  equation  with  two  or  more 
unknowns,  whose  coefficients  are  assumed  integral.  The  problem  is  to 
find  a  set  of  integers,  or  a  family  of  sets,  that  satisfy  the  equation. 

The  classical  way  of  solving  an  equation  with  two  variables  is  by 
applying  certain  recursive  relationships  to  the  quotients  and  remainders 
obtained  from  the  Euclidean  Algorithm,  when  applied  to  the  coefficients 
of  the  two  variables.  This  results  in  two  expressions  for  the  unknowns, 
in  terms  of  one  arbitrary  integer  variable.  An  equation  with  n  unknowns 
(m  >  2)  is  solved  by  solving  a  sequence  of  equations  in  two  variables 
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(method  of  repeated  reductions).  The  final  general  solution  involves 
n  -  1  arbitrary  integer  variables.  (See  ref.  [24],  Chapter  III). 

A  system  of  m  linear  diophantine  equations  with  n  unknowns, 

(m  <  n),  is  solved  by  obtaining  the  general  solution  of  one  of  them  (in 
terms  of  n  -  1  arbitrary  integer  variables)  and  by  introducing  the 
resulting  expressions  into  a  second  equation;  the  general  solution  of  this 
second  equation  will  now  involve  n  -  2  arbitrary  variables.  The  process 
is  repeated  until  every  equation  is  considered.  The  final  solution  for  the 
n  unknowns  will  involve  n  -  m  arbitrary  integer  variables. 

General  Solution  of  Systems  of  Equations 

We  propose  here  a  more  direct  method  of  solution  of  systems  of 

linear  diophantine  equations.  We  will  consider  a  tableau  with  m  +  n 

rows,  where  the  first  m  rows  correspond  to  the  m  equations  and  will 

have  associated  with  them  m  zero-variables  s,  ,  .  .  .  ,  s  .  The  last  m 

1  m 

rows  will  correspond  to  the  n  unknowns  x, ,  .  .  .  ,  x  ,  and  these  will  be 

in 

considered  unsigned  variables  (in  the  next  section  we  will  study  the  non¬ 
negative  solution,  here  we  want  the  general  sciution). 

The  method  is  to  apply,  to  the  first  row,  the  row  reduction 
procedure  of  lemma  3.  3,  until  there  is  only  one  non-zero  element  besides, 
perhaps,  the  zero^  element  (a  ).  Let  the  non-zero  element  be 

X  ,  u 

a,  . ,  j  0  .  Then  we  know  from  lemma  3.  2  that  0  <  a.  „  <  1  a  I  .  If 
l.J  1.0  '  1,  j" 
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a  >  0  ,  then  equation  1  does  not  have  any  solution  in  integers.  (We 
X  f  u 

would  have  s,  =  a,  _  -  a,  .  t.  ,  but  there  is  no  integer  value  of  t.  that 
1  1, 0  1,  j  j  J 

makes  s,  equal  to  zero  because  a,  „  is  not  a  multiple  of  a,  .}.  However, 

1  H  1,0  *  1,J 

if  a^  q  =  0  ,  this  means  that  t.  is  a  zero- variable  and  column  j  may  be 
erased,  thus  reducing  the  size  of  the  tableau.  The  procedure  is  repeated 
with  the  next  row  (now  with  only  n  -  1  variables),  and  the  next,  until 
every  one  of  the  m  rows  has  been  reduced,  and  we  are  left  with  n 
rows  representing  the  general  solution  of  the  original  m  equations.  The 
procedure  is  guaranteed  to  end  in  a  finite  number  of  steps  (see  lemma 
3.3).  For  an  alternate  procedure  isee  Appendix  B. 
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t  q  is  another 

zero- variable. 

The  general 

solution  is 

X1  = 

-  1  -  44  k 

X2  = 

2  +  50  k 

where  k  is  an 

X3  = 

1  +  27  k 

arbitrary  integer 

Analysis  of  Congruences 

Another  important  part  of  number  theory  is  the  solution  of  algebraic 
congruences.  A  linear  congruence  can  be  expressed  as  a  linear  diophantine 
equation.  For  instance  a  x  =  b  (mod  c)  is  the  same  as  the  diophantine 
equation:  a  x  +  c  y  =  b  .  Both  the  problem  variable  x  ,  and  the  "modular" 
variable  y  ,  are  unsigned. 

A  system  of  linear  congruences  can  be  solved  by  the  method  given 
above  for  the  solution  of  systems  of  linear  diophantine  equations.  One 
special  feature  is  that  the  value  of  the  modular  variables  is  of  no  interest, 


therefore,  we  only  need  rows  corresponding  to  the  congruences  and  to 
the  problem  variables. 


5.  3  Non-Negative  Solutions  of  Diophantine  Equations 


The  problem  of  finding  non-negative  solutions  (i.  e.  solutions  in 
non-negative  integers)  of  systems  of  linear  diophantine  equations  is 
also  studied  in  number  theory.  The  problem  may  be  approached  by  means 
of  integer  LP  as  will  be  shown  in  the  following  paragraphs. 


As  in  the  previous  section,  we  will  consider  a  tableau  with 
m  +  n  rows.  The  first  m  rows  correspond  to  the  m  equations  and 
have  m  zero-variables  associated  with  them.  The  last  n  rows 
correspond  to  the  n  unknowns,  and  these  will  now  be  constrained  to  be 
non-negative.  The  first  row  is  taken  at  the  same  time  as  objective 
function  and  as  a  non-negative  constraint;  the  idea  is  to  minimize  a^ 
until  it  becomes  zero  (if  this  is  possible).  But  in  doing  this  we  pivot  in 
such  a  way  that  the  variables  remain  non-negative.  Once  a^  has 
become  zero  we  may  use  the  row  reduction  procedure  of  lemma  3.2, 
until  there  is  only  one  non-zero  element  a^,  j  #  0  in  row  1.  The 
parametric  variable  associated  with  column  j  should  be  a  zero-variable 
so  we  can  now  reduce  the  tableau.  We  repeat  m  times  this  procedure 
of  reducing  rows  and  reducing  the  tableau. 
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The  result  will  be  a  tableau  with  n  rows  and  n  -  m  +  1  columns 

til  til 

(n  -  m  parametric  variables  plus  the  zero  column).  The  zero  column 
will  give  one  non-negative  solution  of  the  original  system  of  equations. 

If  we  are  interested  in  finding  the  lexico-smallest  non-negative  solution, 
we  may  apply  our  basic  algorithm  in  order  to  minimize  the  solution  of  the 
reduced  tableau  (taking  into  account  that  the  n  rows  correspond  to  non¬ 
negative  variables).  When  every  column  becomes  lexico-negative  we 
would  have  obtained  the  lexico-smallest  non-negative  solution  of  the 
original  system  of  equations. 

The  finite  convergence  of  this  procedure  is,  of  course,  dependent 
on  the  finite  convergence  of  the  basic  algorithm. 

If  one  is  interested  in  a  positive  solution,  it  is  sufficient  to  state 

the  problem  in  terms  of  new  variables  x!  ,  where  x!  =  x.  -  1  ,  and 

111 

require  that  the  x!  be  non-negative. 
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CHAPTER  6 


GEOMETRIC  considerations 


6.  I  The  Restricted  Polyhedron  and  the  Primal  Method 


Any  LP  problem  i3  given  by  a  set  of  linear  constraints  and  a  linear 
objective  function  to  be  optimized.  The  set  of  constraints  defines  the 

original  convex  polyhedron, and  it  can  be  shown  that  one  of  the  vertices 

| 

of  this  polyhedron  corresponds  to  an  optimal  solution,  if  an  optimal 

solution  exists.  For  the  continuous  (non-discrete)  LP  problem,  any 

. 

point  on  the  polyhedron  or  in  its  interior  will  be  a  feasible  point.  The 
primal  simplex  method,  through  successive  pivoting,  advances  from 
0  vertex  to  vertex  until  an  optimal  vertex  is  found;  and  it  does  this  in  such, 
a  way  that  the  value  of  the  objective  function  is  non-decreasing  as  the 
1  algorithm  moves  from  one  vertex  to  the  next. 


In  the  case  of  a  discrete  LP  problem  (see  section  1.  1),  we  also 
have  the  original  polyhedron,  but  it  is  no  longer  true  that  any  point  on 
its  surface  is  a  feasible  point.  However,  there  exists  one,  and  only  one, 
restricted  convex  polyhedron  that  is  contained  in  the  original  polyhedron 
and  such  that:  a)  every  vertex  of  the  restricted  polyhedron  is  a  feasible 
point  (i.e.  it  satisfies  both  the  constraints  and  the  discreteness 
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requirements),  and  b)  there  are  no  feasible  points  outside  the  restricted 
polyhedron.  (Notice  that  this  does  not  imply  that  every  point  on  or 
within  the  restricted  polyhedron  is  a  feasible  point. )  The  idea  of  the  prim 
algorithm  is  to  advance  (in  a  simplex-like  manner)  from  a  vertex  of  this 
restricted  polyhedron  to  another  vertex  of  the  same  polyhedron  until  the 
optimum  is  reached.  By  using  the  lexicographical  simplex  method  the 
algorithm  always  advances  to  another  vertex  that  has  a  larger  lexico¬ 
graphical  value  (or  smaller,  if  we  are  minimizing). 

In  order  to  constrain  our  computational  "walk"  to  the  restricted 
polyhedron,  we  have  to  introduce  some  additional  inequalities  or  "cuts". 
These  cuts  serve  only  as  guiding  walls  and  as  stopping  barriers  to 
constrain  our  walk,  but  they  need  not  actually  correspond  to  any  of  the 
planes  that  would  define  the  restricted  polyhedron.  In  the  following 
sections  we  will  develop  these  ideas  further. 

6.2  The  Parametric  Variables  as  a  System  of  Coordinates 

The  following  discussion  has  to  do  with  our  primal  algorithm  for 
integer  LP. 

The  original  constraints  of  the  problem  are  defined  in  terms  of 
x, ,  .  .  .  ,  x  ,  and  are  hence  defined  in  an  n-dimensional  real  space  Rn 

In 

having  these  variables  as  coordinates.  But  by  means  of  linear  trans¬ 
formations  we  can  express  the  constraints  in  terms  of  different  sets  of 


coordinates.  This  is  precisely  what  we  accomplish  by  pivoting:  we 

express  all  the  constraints  in  terms  of  different  sets  of  parametric 
k  k 

variables  t. t  .  It  is  possible  to  express  the  constraints  in  terms 
In  r 

of  n  arbitrary  variables  as  long  as  the  system  of  coordinates  they  define 
covers  the  complete  space  Rn  .  However,  by  obtaining  the  parametric 
variables  as  the  result  of  generating  Gomory-cuts  (remember  that  the 
parametric  variables  are  the  slack  variables  of  the  Gomory  inequalities) 
we  achieve  the  following  results: 

a)  The  n  coordinate  planes,  corresponding  to  the  parametric 
variables,  intersect  at  one  of  the  vertices  of  the  restricted  polyhedron. 
This  vertex  is  precisely  the  one  represented  by  the  current  solution 

(i.  e.  the  point  defined  by  the  current  values  of  x  ,  .  .  .  ,  x  ). 

1  n 

b)  The  restricted  polyhedron  is  entirely  contained  in  the  non¬ 
negative  orthant  of  the  space  defined  by  the  parametric  variables,  as 
can  be  assured  by  the  fact  that  these  variables  can  only  take  non-negative 
values  whenever  we  have  a  feasible  (integer)  solution,  ^rhit,  characteristic 
is  specially  important  for  proving  optimality.  Notice  also  that  this 
characteristic  is  specially  important  for  proving  optimality.  Notice  also 
that  this  characteristic  does  not  imply  that  the  original  polyhedron  itself 
will  be  entirely  contained  in  the  non-negative  orthant.  This  is  not  the 
case,  for  instance,  if  the  current  (integer)  solution  is  not  also  a  vertex 

of  the  original  polyhedron. 


c)  There  is  a  one-to-one  mapping  of  the  lattice  of  integer  points 
defined  by  x^.  .  .  ,  and  the  lattice  of  integer  points  defined  by  the 
parametric  variables.  Among  other  things,  this  assures  that  whenever 
the  parametric  variables  have  integer  values  so  will  the  original  variable 
have  and  vice  versa.  This  one-to-one  correspondence  can  be  proved  by 
showing  that  the  determinant  of  t 

in  terras  of  the  parametric  variables  is  always  +1  or  -1  (D  is  formed 


that  helps  to  complete  a  (n  +  1) 


he  matrix  D  that  expresses  the  x^'  s 


with  tne  n  rows  labeled  x  , ....  x  plus  an  auxiliary  row  -1  =  -1 


n 


by  (n  +  1)  matrix.  We  will  initially 


have  |  D|  =  "t  1  ,  and  our  pivoting  formulas  result  in  column  operations 

| 

that  do  not  change  the  absolute  value  of  |  D|  . ) 

In  geometric  terms,  whenever  there  is  a  breakthrough  in  a  certain 
column,  this  implies  that  the  intersection  of  the  n  -  1  planes  correspond 

■  t  l 

to  the  other  parametric  variables,  coincides  with  one  of  the  edges  of  the 

( 

restricted  polyhedron. 

I;  ,  -  |  •• 

Geometric  visualizations  of  some  of  the  aspects  of  the  problem, 
although  difficult  to  accomplish,  provide  additional  insight  into  the  workin> 
of  the  algorithm. 


An  Important  Observation  about  Gomory  Cuts 

Another  important  observation  and  one  that,  to  the  best  of  our 
knowledge,  has  not  explicitly  been  made  by  Gomory  or  by  anybody  else, 


is  the  following: 


Any  Gomory  cut  generated  from  a  certain  row  i  (constraint  i  ) 
will  be  such  that  it  will  be  satisfied  by  every  non-negative  integer  point 
(non-negative  with  respect  to  the  current  set  of  parametric  variables) 
that  satisfies  constraint  i  .  However,  the  Gomory  cut  can  be,  and  most 
often  will  be,  unsatisfied  by  some  feasible  integer  points  (feasible  with 
respect  to  constraint  i  )  that  are  non-negative  wit  .  respect  to  the 
original  variables.  The  Gomory  cut  generated  from  row  i  is  then  also 
a  function  of  the  rows  that  have  been  used  to  generate  the  current  set  of 
parametric  variables. 

6.  3  Inverse  Pivoting 

By  successive  pivoting  operations  we  express  the  original  variables 
(Xj,  .  .  .  ,  x^)  in  terms  of  the  parametric  variables.  If  we  wanted  to  express 
the  parametric  variables  in  terms  of  the  original  variables,  we  would 
have  to  invert  the  matrix  that  expresses  the  previous  relationship. 

However,  if  we  choose  to  carry  this  inverse  matrix  along  with  our 
regular  tableau,  the  updating  operation  is  a  simple  one,  and  there  is  the 
added  advantage  of  having  obtained  the  inverse  for  every  intermediate 
tableau  and  not  just  for  some  specific  tableau.  The  updating  operation 
will  be  called  inverse  pivoting,  and  it  will  be  defined  in  the  following 
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paragraphs 


Let  us  establish  a  matrix  notation  that  is  slightly  different  from  thi 


of  (2.1.5): 
Let 


82 


and 


(6.  3.  3) 


The  square  matrix  P  is  the  transformation  matrix  that  expresses 

the  pivoting  operation.  Because  we  pivot  on  the  element  of  the  row 

k  k  k  k 

(brt  ,  b,  ,  . .  . ,  b  ),  and  because  b.„  =  1  ,  we  have  the  following: 

0  1  n  j  u 


hk 

•  jO-1 


-l  -b! 


(6.  3.  4) 


From  (6.  3.  3)  we  have 


_2  1  1  o  o  —.1 

D=DP=DPP 


using  (6.  3.2)  we  obtain,  in  general 


_o  o  _1  _k- 1  k 

X  =  D  P.  P  .  .  .  P  T 
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from  which 


Tk  =  (D°  P°  P1.  .  .  P1""1)"1  X  =  (Pk_1)"1.  .  .(P1)'1  (F°f 1  (D0)'1 
if  we  define  E°  =  (D°)_1  =  D°  =  -  I  (6.  3.  6) 

and 

Ek+1  =  (PV1  Ek  ,  k  =  0,  1 ,  . .  .  (6.3.7) 


then 

Tk  =  Ek  X  (6.  3.  8) 

Jq 

Matrix  P  is  an  elementary  matrix,  and  because  row  jO  has  a  -  1  in 
the  diagonal,  it  turns  out  that  it  ^s  its  own  inverse,  as  can  be  easily 
verified  by  multiplication.  Equation  (6.  3.  7)  can  then  be  expressed  as 

k+i  k  ,  .  .n 

e  .  =  e. .  ;  for  x  #  iO 

ij  V 

j  =  0,  1,  .  .  . ,  n  (6.  3.  9) 

n 

k+1  V  ,k  k 

ejo,j  "  Z  j  ejo,j  * 

i=0 

k 

We  may  carry  matrix  E  in  our  computations  and  perform  the 
addition  of  rows  expressed  in  (6.  3.  9),  but  it  is  mnemonically  easier  to 
carry  its  transposition  and  to  perform  the  operation  on  the  corresponding 
columns.  Thus,  under  each  column  of  the  regular  tableau  we  will  have 
another  column  that  expresses  the  corresponding  parametric  variable  in 
terms  of  the  original  variables. 
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This  technique  of  inverse  pivoting  is  specially  useful  for  research 
or  whenever  one  is  interested  in  learning  how  the  successive  Gomory-cuts 
look  like  in  the  original  space. 

6.4  Natural  Inequalities 

Let  us  consider  the  following  inequality: 

8  -  7  x£  +  5  x3  «  19  (6.  4.  1) 

Where  x  ,  x  and  x  are  non-negative  integer  variables.  If  we 

1  M  J 

let  x_  =  x  =  0  then  (6.  4.  1 )  would  imply  x  <  2  <  [19/8].  Similarly  if 

Ct  j  1 

x  =  x  =  0  w  would  have  x  <  3  ^[19/5],  On  the  other  hand,  if  we 

1  M  D 

generate  a  Gomory-cut  from  (6.  4.  1)  with  p  =  8  and  another  one  with 
p  -  5  ,  we  will  have: 


With 

p  =  8: 

xi  -  *2  '2 

(6.4.2) 

With 

p  =  5: 

*1  '  2  x2  +  x3  *  3 

(6.4.  3) 

We  see  that  (6.  4.  2)  strictly  binds  x^  (assuming  x^  =  X3  =  0)  to  the 
integer  value  that  (6.  4.  1)  only  implies;  and  (6.  4.  3)  binds  x^  in  a 
similar  way.  We  say  that  (6.  4.  2)  is  a  natural  inequality  of  (6.  4.  1)  for 
Xj  ;  and  (6.  4.  3)  is  a  natural  inequality  of  (6.  4.  1)  for  x^  . 

In  general,  if  we  have  an  inequality: 

a  x  +  a  x  +.  .  .  +  a  x  « a  ;  (6.4.4) 

1  2  2  n  n  0 
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and  if  has  the  same  sign  as  a^  ,  then  the  Gomory-cut  generated  from 
(6.  4.  4)  with  p  =  j  a.|  is  said  to  be  the  natural  inequality  of  (6.  4.  4)  with 
respect  to  x^  . 

All  the  Gomory-cuts  that  are  used  in  our  primal  algorithm  are 
natural  inequalities.  (They  are  "zero  natural  inequalities"  or  "non-zero 
natural  inequalities"  according  to  whether  they  are  "ze-  o-cut"  or  not.  ) 

6.  5  Deepest  Cuts  and  Hierarchical  Rules 

We  define  a  deepest  cut  as  any  of  the  hyperplanes  that  correspond  to 
the  several  faces  of  the  restricted  polyhedron.  A  deepest  cut  in  this 
sense,  is  different  than  the  deepest  cuts  mentioned  in  [5],  [8]  and  [11], 
where  the  concept  is  more  that  of  "deepest"  in  a  certain  direction. 

The  number  of  faces  of  the  restricted  polyhedron  may  be  smaller 
than,  equal  to,  or  larger  than  the  number  of  faces  of  the  original  polyhedro 
as  may  be  verified  by  drawing  simple  two  dimensional  examples. 

A  sufficient,  but  not  necessary,  condition  for  a  certain  parametric 
variable  t.  to  be  a  "deepest-cut  variable"  is  if  there  is  a  breakthrough 
in  every  one  of  the  other  columns.  This  is  because  it  shows  that,  besides 
the  current  vertex,  there  are  other  n  -  1  vertices  of  the  restricted 
polyhedron  that  lie  on  the  hyperplane  =  0  ,  thus  determining  it  as  a 
deepest  cut. 
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In  general,  it  is  difficult  to  have  a  tableau  in  which  every  parametric 
variable  is  a  deepest  cut  variable.  This  is  because  by  using  an  "all¬ 
integer"  method,  we  are  limiting  ourselves  in  our  choice  of  sets  of 
parametric  variables.  (If  we  take  any  n  deepest  cuts  that  define  a  vertex 
of  the  restricted  polyhedron  and  if  we  were  to  use  this  set  as  a  system  of 
coordinates  to  express  the  original  constraints,  we  would  have  to  use,  in 
general,  fractional  coefficients.  ) 

However,  it  is  conceivable  that  one  of  the  parametric  variables 

(say  t  )  might  correspond  to  a  deepest  cut;  and  that  the  projection  of  a 

second  one  (say  t  ),  might  be  a  deepest  cut  of  the  (n-1)  dimensional 

n- 1 

projection  of  the  restricted  polyhedron  over  the  hyperplane  t  =  0  ;  and 

n 

subsequently,  t  could  be  a  (n-2)  dimensional  deepest  cut,  and  so 

Xl-C 

forth.  This  is  the  motivation  for  the  hierarchical  rules  that  were  given 
in  section  4.  3.  A  deeper  study  of  this  aspect  could  lead  to  better 
hierarchical  rules. 

6.6  Geometric  Considerations  of  Two-Dimensional  Convergence 

In  section  4.  2  we  developed  a  proof  of  finite  convergence  of  the 
basic  algorithm  when  applied  to  two-dimensional  problems.  In  this 
proof,  an  algebraic  concept  of  dominance  of  rows  was  used.  We  will  now 
present  the  geometric  motivation  of  this  concept. 

Let  us  consider  the  two-dimensional  space  as  given  in  terms  of 
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We  will  define  the 


the  current  parametric  variables  t  and  t^  . 
primary  region  as  the  region  given  by  t^  5  0  ,  t^  >  0  ,  z  $  a^  ^  .  That 
is,  every  point  in  this  region  is  non-negative  and  would  give  z  a  value 
not  smaller  than  its  current  value.  Clearly,  the  optimal  solution, 
(assuming  it  exists),  is  to  be  found  in  the  primary  region. 

If  a  certain  constraint  (row)  is  satisfied  by  every  point  of  the 

m 

primary  region,  the  constraint  is  said  to  be  dominated;  if  it  is  not 
satisfied  by  a  subset  of  this  region  it  is  said  to  be  non -dominated.  This 
geometric  definition  of  dominance  coincides  with  the  algebraic  definition 
given  in  section  4.2,  as  it  can  easily  be  shown. 

When  the  primary  region  does  not  include  any  point  of  the  positive 
orthant  (i.  e.  any  point  such  that  t  >  0  ,  t  >  0  )  it  means  that  the 
objective  function,  z  ,  has  reached  its  optimal  value,  and  the  present 
solution  is  optimal  (although  there  might  be  other  optimal  solutions). 

Whenever  we  generate  a  cut  for  pivoting,  it  will  be  non-dominated 
and,  once  we  pivot,  the  cut  will  become  one  of  coordinate  axis  and  the 
new  primary  region  will  be  a  subset  of  the  previous  one.  This  will  occur 
whether  we  have  a  zero-  or  a  non-zero-cut.  If  we  have  a  non-zero-cut, 
there  will  be  a  shift  of  the  line  z  =  an  (when  a  >  a  )  so  that  the 

V  )  V  W  >  V  W|  V 

primary  region  will  be  reduced  further. 

What  the  proof  of  section  4.  2  shows,  is,  in  geometric  terms,  that 
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as  we  introduce  new  cuts,  the  primary  region  diminishes  consistently 
until  the  positive  orthant  (in  terms  of  the  current  set  of  parametric 
variables)  is  completely  excluded.  At  this  point  optimality  is  proved. 

In  other  words,  every  time  we  pivot,  the  system  of  coordinates  is 

-f 

translated  from  one  vertex  to  another  and/pr  rotated  around  a  vertex  of 
the  restricted  polyhedron,  so  that  eventually  there  is  no  better  point  in 
the  non-negative  orthant  of  the  (current)  system  of  coordinates. 
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CHAPTER  7 


SUGGESTED  EXTENSIONS 


In  the  first  section  of  this  chapter  we  present  the  concept  of  a 
"Tight"  Algorithm  which  would  be  similar  to  our  basic  algorithm  except 
that  the  parametric  variables  would  always  correspond  to  deepest  cuts.  In 
the  second  section  we  introduce  the  notion  of  fractionalization  of  variables, 
and  we  advance  some  ideas  as  to  how  this  concept  could  be  used  to  develop 
a  new  approach  for  solving  mixed  integer  LP  problems. 

7.  1  A  "Tight"  Algorithm 

Further  study  of  the  Gomory  cuts  in  the  context  of  the  primal  feasible 
algorithm  should  lead  to  a  procedure  for  generating  some  of  the  deepest 
cuts  that  define  any  specific  vertex  of  the  restricted  polyhedron.  By  using 
inverse  pivoting  these  deepest  cuts  may  be  expressed  in  terms  of  some 
initial  set  of  parametric  variables. 

To  be  more  specific,  let  us  assume  that  we  have  our  tableau  (with 
an  initial  feasible  solution),  and  that  every  parametric  variable  t.  is  a 
deepest  cut  variable.  This  means  that  there  is  a  breakthrough  in  every 
column  (some  are  in  the  maximizing  direction,  some  in  the  minimizing 
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direction).  If  we  are  maximizing  we  could  choose  any  negative  column 
and  we  would  be  able  to  obtain  a  non-zero-cut  that,  upon  pivoting,  would 
take  us  to  another  vertex.  This  non-zero-cut  will  not  necessarily  be  a 
deepest  cut.  However,  we  might  be  able  to  find  a  "deepest  cut  procedure" 
so  that,  in  fact,  we  obtain  the  expression  of  that  deepest  cut  which  would 
not  only  move  to  the  next  vertex,  but  the  tableau  would  again  be  entirely  in 
terms  of  deepest  cut  variables. 


The  tableau  would  have  rational  coefficients,  rat  her  than  integer 

I  .  i 

*  I,  |i 

ones,  but  by  appropriately  changing  the  scale  of  some  or  all  of  the  row 
variables,  we  should  be  able  to  keep  the  new  coefficients  in  terms  of 


integers. 


A  "Tight"  Algorithm  could  have  two  special  advantages:  in  the  first 

place,  it  might  turn  out  to  be  computationally  more  efficient  than  the 

* 

Basic  Algorithm  (i.  e.  it  might  be  an  "accelerated"  version  of  the  Basic 
Algorithm);  in  the  second  place,  the  fact  that  at  every  ^vertex  the  constraints 
may  be  expressed  in  terms  of  a  uniquely  defined  set  of  parametric 
variables  (i.e.  the  variables  corresponding  to  the  deepest  cuts  that  define 
that  vertex)  could  turn  out  to  have  important  consequences  for  the  study  of 
duality  in  integer  LP. 
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7.  2  Fractionalization  of  Variables  and  Mixed  Integer  LP 

Two  of  the  reasons  for  the  use  of  all-integer  algorithms  in  integer 

LP  are:  first,  to  obtain  exact  solutions;  and  second,  to  use,  at  least 

implicitly,  the  number  theoretic  properties  of  the  coefficients.  However, 

it  should  be  pointed  out  that  the  generation  of  Gomory  cuts  is  valid 

whether  the  coefficients  are  integer  or  not.  For  example,  if  all  the 

coefficients  of  the  tableau  (excepting  the  last  n  rows  which  give  the 

integer  variables  x^  ,  j  =  1,  . . . ,  n  in  terms  of  the  parametric  variables 

t.  )  were  not  integral,  our  primal  feasible  algorithm  should  still  work.  The 
1 

slack  variables  might  not  have  integer  values,  but  the  x^ ' s  will  always 
have  integer  values. 

We  define  fractionalization  of  a  variable  as  the  process  of  replacing 

a  parametric  variable  t.  by  another  variable  t!  such  that  t.  =  t'./k  , 

1  J  ”  J  J 

where  k  is  some  positive  integer.  This  is  equivalent  to  dividing  every 
element  in  column  j  by  k  .  We  could  either  express  these  new  elements 
as  rational  numbers,  or  as  decimal  fractions;  or  we  could  multiply  every 
element  of  the  tableau  by  k  ,  thus  eliminating  fractions  by  changing  the 
scale  of  the  row  variables. 

The  result  of  this  fractionalization  is  that  the  lattice  of  integer 

points  of  the  new  system  of  coordinates  is  "finer"  (in  the  t. -direction) 

J 

than  the  previous  lattice.  However,  every  point  of  the  old  lattice  is  also 
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a  point  of  the  new  one. 

If  all  the  coefficients  and  constants  of  a  mixed  integer  LP  problem 
are  rational  numbers,  then  every  extreme  jlution  (i.e.  every  vertex  of 
the  restricted  polyhedron)  will  be  in  terms  of  rational  numbers.  It  is 
conceivable  that  a  mixed  integer  LP  problem  could  be  transformed  into  a 
pure  integer  LP  problem  by  proper  fractionalization  of  some  of  the 
variables. 
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APPENDIX  A 


SOME  CONSIDERATIONS  ABOUT  THE  LEXICOGRAPHICAL 

SIMFLEX  METHOD 


A.  I  Proof  of  Optimality 

Lemma  A.  1  If  the  tableau  is  both  primal  feasible  and  dual  feasible, 
the  current  solution  is  optimal. 


Proof:  Let  us  assume  that  we  are  maximizing. 


dual  feasible,  it  then  means  that  a  ^  0  ;  j  =  1,..., 
expression  for  z  is: 


If  the  tableau  is 
n  .  The  current 


n 


j  =  l 


We  can  restrict  the  t. ' s  to  be  non-negative  without  losing  any 

Jq 

feasible  (integer)  solution.  But  if  the  t.  1  s  are  non-negative,  this  means 

k 

that  the  largest  value  of  z  is  precisely  a^  ^  ,  and  therefore,  the  current 
solution,  being  feasible,  is  also  optimal. 


If  we  were  minimizing,  dual  feasibility  would  imply  that 
<0;  j  =  1,  .  .  .  ,  n  and  the  proof  would  follow  in  a  similar  way.  (Q.  E.  D.  ) 


-97- 


A.  2  Proof  of  Lexico-Optimalit\ 


Lemma  A.  2  If  the  tableau  is  both  primal  feasible  and  lexico-dual 
feasible,  the  current  solution  is  lexico-optimal. 

Proof:  Let  us  assume  that  we  are  lexico-maximizing.  If  the  tableau 

is  lexico-dual  feasible,  it  means  that  every  column  (not  considering 

column  zero)  is  lexico-positive;  that  is,  the  first  non-zero  element  of 

each  column  is  positive.  This  means  that  a^  .50,  j  =  1,  .  .  . ,  n  ,  and 

from  lemma  A.  1,  we  realize  that  z  has  attained  its  maximum  value.  We 

have  to  check  whether  the  solution  is  the  lexico-largest  feasible  solution. 

th 

That  is,  we  have  to  see  if  the  h  component  of  the  solution  could  increas 
its  current  value  without  decreasing  the  value  of  any  one  of  the  previous 
components.  Consider  the  following  row: 


8  =  a 

h  h 


0+  J  \  .  (-tk 

,  0  Lj  h,  J  j 


Now,  because  we  constrain  the  t. ' s  to  be  non-negative,  s  can 

J  h 

k  k 

only  increase  if  a,  .  <  0  for  some  j  ,  say  j  =  jO  .  But  if  a,  <  0  , 

n,  j  h, jO 

k 

this  means  that  there  must  be  some  i  <  h  such  that  a.  >  0  ,  because 

i.  jO 

every  column  is  lexico-positive.  Thus,  if  we  increase  t  ,  the  value  of 

J 

s  will  certainly  increase,  but  s.  will  decrease,  and  this  will  result  in 
h  l 

y  a  lexico-smaller  solution.  Therefore,  the  current  solution  is  the  lexico- 


largest. 
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A  similar  argument  would  be  used  if  we  were  lexico-minimizing. 


(Q.E.D.) 


A.  3  The  Usefulness  of  a  Lexicographical  Simplex  Method 

If  we  are  minimizing  with  a  "plain"  simplex  method,  we  consider  for 

pivoting  only  those  columns  j  with  an  .  >  0  ;  whereas,  if  we  are  using  a 

0*  j 

lexicographical  method,  we  will  consider  those  columns  which  are  lexico- 
positive. 


Let  us  consider  the  following  problems: 


min  z  =  x2  "  X3 

with  -  x,  -x,  +  2x,  ^1 
12  3 

*1  ‘  2  *2  +  x3  *  1 
x  ,  x  ,  x  non-negative  integers 

i  w  J 

If  we  try  to  solve  this  example  with  our  basic  algorithm,  and  using 
a  "plain"  criterion  for  considering  pivoting  columns,  we  will  always  have 
a  single  choice  of  pivoting  column  and  a  single  choice  of  generating  row. 
Furthermore,  cycling  will  occur  and  the  optimal  solution  will  not  be 
found. 


On  the  other  hand,  if  we  use  a  lexicographical  criterion  for 
considering  pivoting  columns,  we  will  obtain  the  lexico-optimal  solution 
in  four  iterations.  This  example  shows  that  the  lexicographical  simplex 


-99- 


method,  or  some  other  device  to  stop  cycling,  is  necessary  for  our  has 
algorithm. 
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APPENDIX  B 


LEXICO- MINIMIZATION  AS  A  PROCEDURE 
FOR  IMPLICIT  TABLEAU  REDUCTION 


In  Chapter  3  it  was  shown  how  one  could  take  advantage  of  any 
equalities  in  the  original  problem,  for  reducing  the  size  of  the  tableau 
through  the  elimination  of  zero-variables.  It  turns  out  that  an  equivalent 
result  can  be  obtained  in  some  instances  by  properly  rearranging  rows  and 
then  lexico-minimizing  the  problem. 

More  specifically,  let  us  assume  that  we  have  a  tableau  such  that 
the  only  infeasibilities  correspond  to  some  zero-variables  that  have  non¬ 
zero  values.  We  now  rearrange  the  rows  so  that  all  the  zero- variable 
rows  occupy  the  first  places  (including  those  zero-variable  rows  that  are 
feasible),  followed  by  the  objective  variable  row,  by  the  rows  corresponding 
to  non-negative  variables,  and  by  the  rows  corresponding  to  other  unsigned 
variables  (we  may  have  or  may  not  have  an  objective  variable). 

If  the  problem  is  lexico-minimized  and  there  exists  a  feasible 
solution,  the  first  components  of  the  solution  -  those  corresponding  to  the 
zero-variables  -  will  have  a  value  of  zero.  Furthermore,  because  the 
lexico-optimal  solution  requires  that  every  column  be  lexico-negative, 
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this  implies  that  there  is  no  positive  element  in  the  first  row.  As  a 
matter  of  fact,  in  the  process  of  solution  there  will  be  a  step  where  only 
one  negative  element  will  be  left  in  row  1;  every  other  element  being 
zero.  It  follows  that  the  column  corresponding  to  this  negative  element 
will  never  again  be  chosen  for  pivoting  because  it  is  lexico-negative  and 
there  is  no  way  in  which  ic  can  change  its  lexicographical  sign.  This  is 
what  we  call  implicit  reduction  of  the  tableau.  Similarly,  the  second  row 
will  reach  a  later  step  where  (if  we  leave  out  the  column  implicitly  reduced 
by  row  1)  it  will  have  a  single  negative  element  among  zero  elements. 

This  negative  element  determines  another  negative  column  that  is 
implicitly  reduced,  and  which  will  not  be  chosen  for  pivoting.  A  similar 
thing  may  happen  with  every  zero-variable  row.  The  net  result  is  that  the 
tableau  automatically  acquires  a  triangular  canonical  form. 
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